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The  1977  National  Symposium  of  the  Military  Services  on  Utilization 
of  Peop1e--Re1ated  llesearch,  Developinent,  T^t,  and  Evaluation  (Rbt&E) 
was  held  in  San  Diego,  California  on  14-17  June  1977.  It  was  sponsored 
and  conducted  by  the  Navy  Personnel  Research  and  Development  Center 
(NAVPERSRANDCEN) , San  Diego,  with  the  support  of  the  Human  Interaction 
Research  Institute  (HIRI),  Los  Angeles,  under  Contract  No.  N00123-76-C- 
0174.  The  general  chairman  was  Dr.  Franklin  F.  Sands,  NAVPERSRANDCEN; 
and  the  faculty  chairman.  Dr.  Edward  M.  Glaser,  HIRI.  There  were  120 
participants,  including  representatives  from  the  Department  of  Defense, 
other  government  and  private  agencies,  and  universities  (see  Appendix  A). 

The  purpose  of  the  symposium  was  to  assess  the  problems  and  needs  of  human 
resources  RDT&E,  principally  within  the  military  establishment,  and  to 
formulate  recommendations  for  improved  utilization  of  the  RDT&E.  Since  most 
of  the  participants  were  very  familiar  with  issues  of  research  utilization, 
the  program  agenda  and  work  plan  (Appendices  B and  C)  were  designed  to  permit 
a high  degree  of  participation  by  attendees.  This  was  done  by  providing  not 
only  prepared  presentations  but  also  the  opportunity  for  conferees  to  meet  in 
small  groups  to  address  substantive  issues  and  to  reconvene  in  plenary  sessions 
to  integrate  problems,  needs,  and  recommendations.  This  process  was  greatly 
facilitated  by  symposium  faculty  members  (Appendix  D). 

The  symposium  was  organized  into  the  following  nine  sessions: 

1.  Invited  Addresses:  Major  addresses  by  key  DoD  policymakers. 

2.  Military  Service  Group  Meetings:  Held  to  identify  barriers  to 
utilization  and  recommend  possible  solutions. 

3.  Reflections  from  the  Operational  Community:  Central  issues  in 
utilization  as  seen  by  the  operational  common i ty . 

4.  State-of-the-Art  in  Research  Utilization  Within  Other  Federal 
Agencies:  Experiences  of  nonmilitary  agencies  in  dealing  with 
research  utilization. 

5.  Overall  Research  Utilization  Perspective:  Summary  of  recurrent 
theme's  from  first  portion  of  symposium. 

6.  Technology  Transfer  Pane]  Discussion:  Comments  on  transfer  of 
technology  from  federal  laboratories  to  state/city  governments. 

7.  Case  Problem  Analysis:  Discussions  of  prepared  case  problems  in 
utilization  (summaries  included  in  Appendix  E). 

8.  Selected  Concept  Papers:  Presentation  of  two  papers  that  highlight 
util ization  issues. 

9.  Policy  Recommendations  for  Improved  Research  Utilization:  Small 
group  meetings  to  develop  recommendations  for  improved  RU. 


V 


The  proceedings  provided  by  this  report  include  (1)  an  edited  transcript  of 
the  presentations,  small  group  reports  in  plenary  session,  and  open  discus- 
sion-organized by  the  above  nine  sessions,  (2)  summaries  (on  blue  paper) 
of  the  nine  sessions,  (3)  a synthesis  (on  green  paper)  integrating  the  nine 
summaries  and  (4)  the  appendices.  Appendix  F provides  a sunmary  of  symposium 
evaluation  comments  provided  by  participants. 

Special  appreciation  is  extended  to  the  following  invited  speakers  (Session  1) 
who  provided  a provocative  thrust  and  challenge  to  symposium  deliberations: 

1.  RADM  Joseph  Metcalf,  III,  USN,  Assistant  Chief  of  Naval  Personnel  for 
Financial  Management  and  Management  Information. 

2.  Dr.  Bernard  Rostker,  Principal  Deputy  Assistant  Secretary  of  the  Navy 
for  Manpower  and  Reserve  Affairs. 

3.  COL  Henry  L.  Taylor,  USAF,  Office  of  the  Director,  Defense  Research 
and  Engineering. 

Appreciation  is  also  extended  to: 

1.  Dr.  Edward  Glaser  of  HIRI  and  to  members  of  the  Symposium  Steering 
Comnittee  and  Symposium  Faculty  (listed  in  Appendix  D),  for  their  assistance 
in  planning  and  conducting  the  symposium. 

2.  Ms.  Linda  Culligan  and  Dr.  Paul  Greenberg  of  HIRI,  for  their  assistance 
in  preparing  the  symposium  proceedings. 

3.  The  following  NAVPERSRANDCEN  personnel,  for  their  contributions  in 
planning  and  execution  of  logistics  in  support  of  the  symposium:_ 

Mrs.  Loretta  T.  Teague,  Mr.  Robert  F.  Turney,  LCDR  David  C.  Emerson, 

Mr.  Walter  M.  Spencer,  Mr.  Robert  Harrigan,  and  ETl  Gerald  G.  Boykin,  USN. 


J.  J.  CLARKIN 
Commanding  Officer 
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INVITED  ADDRESSES 


SUMMARY 


The  task  of  the  symposium  partiaipants — to  assess  the  problems  of 
utilization  of  human  resources  RUT&E  in  the  military  environment 
and  to  formulate  policy  recommendations  for  improving  utilization — 
was  outlined  in  the  welcoming  remarks  by  Dr.  Frank  Sands,  the 
symposium  general  chairman,  and  CAPT  James  Clarkin,  USD,  Commanding 
Officer  of  the  host  organization.  Dr.  Sands  emphasized  the  growing 
pressure  at  the  federal  level  to  increase  the  rate  of  R&D  product 
use.  CAPT  Clarkin  urged  that  members  of  the  R&D  community  pay 
particular  attention  to  utilization  barriers  in  their  own  community, 
and  be  willing  to  compromise  as  they  work  with  the  sponsors  and 
users  to  achieve  greater  utilization. 

The  charge  and  challenge  to  the  conferees  posed  by  Dr.  Sands  and 
CAPT  Clarkin  was  fleshed  out  in  substantial  detail  in  magor  addresses 
by  three  key  DoD  policymakers.  The  keynote  speaker,  RADM  Joseph  Metcalf  III, 
USD,  Assistant  Chief  of  Naval  Personnel  for  Financial  Management 
and  Management  Information,  concluded  that  the  essence  of  the  utili- 
zation problem  is  failure  to  consistently  and  effectively  plan  for 
implementation.  He  cited  three  fundamental  reasons  for  this: 
resistance  to  change,  conflict  of  interest  between  the  research  and 
user  communities,  and  the  difficulty  in  establishing  meaningful 
criteria  for  measuring  the  success  of  human  resources  research  and 
utilization  activities.  He  pointed  to  the  need  for  better  meth'>ds 
for  managing  R&D  planning,  including  the  explicit  assignment  of 
responsibility  for  implementation  planning  in  order  to  improve 
accountability  for  utilization. 

COL  H.  L.  Taylor,  VSAF,  Office  of  the  Director,  Defense  Research 
and  Engineering,  offered  a perspective  on  RDT&E  utilization  from 
the  vantage  point  of  DDR&E.  COL  Taylor  reviewed  a series  of  magor 
historical  factors  that  have  intensified  the  pressure  for  improved 
utilization  of  people-related  research  within  the  military  services. 

The  factors  include: 

. Growing  interest  by  the  Congress  in  the  relevance  of  research, 
including  the  Mansfield  Amendment,  which  stated  that  all  defense 
R&D  must  have  a direct  and  apparent  relationship  to  the  DoD 
mission. 

. The  Ginzberg  Management  Report. 

. The  House  Appropriation  Cormittee  report  on  the  FY  76  budget 
request,  recommending  a SO  percent  reduction  in  five  critical 
program  areas  of  human  resources  R&D. 

. The  2976  Defense  Science  Board  Task  Force  on  Training  Technology. 

. The  April  1977  GAO  report,  which  made  a series  of  recormendations 
to  DoD  for  improvement  in  managing  the  utilization  of  human 
resources  R&D. 


COL  Taylor  aonoluded  that  without  better  and  more  visible  utilization 
of  research  results y support  for  R&D  in  the  human  resources  area  will 
dry  upy  and  this  will  have  a critical  impact  on  combat  readiness. 

The  banquet  speaker  was  Dr.  Bernard  Rostkery  Principal  Deputy  Assistant 
Secretary  of  the  Navy  for  Manpower  and  Reserve  Affairs.  Dr.  Rostker 
asserted  that  human  resources  R&D  should  be  structured  to  address  policy 
issues  and  operational  problems y and  that  much  of  the  work  fails  to  do 
so  and  is  neither  relevant  nor  useful.  He  further  contended  that  com- 
nrunication  is  poor  between  researchers  and  users  and  between  researchers 
and  policymakers y that  there  is  not  enough  follow-through  from  research 
to  applicationy  and  that  this  is  largely  the  responsibility  of  the  R&D 
community.  He  concluded  that  the  planning y programming  and  budgeting 
of  R&D  should  be  realigned  so  as  to  put  control  in  the  hands  of 
policymakers. 


INVITED  ADDRESSES 


SYMPOSIUM  INTRODUCTION 


Dr.  Franklin  F.  Sands 
Applications  Support  Office 
Navy  Personnel  R&D  Center 


I would  like  to  welcome  you  here  with  the  hope  that  this  research 
utilization  symposium  proves  to  be  interesting  and  successful,  and 
some  useful  items  can  be  taken  away  from  this  meeting.  It  is  our 
goal  that  the  integrated  summaries  which  we  will  be  developing 
later  on  in  the  week  will  have  a positive  impact  on  current  and 
future  research  utilization  policy. 

As  you  probably  know,  federal  support  for  RSD  has  grown  to  the  level 
of  about  22  billion  dollars  per  year.  Of  this  amount,  the  Department 
of  Defense  accounts  for  approximately  12  billion,  of  which  about 
2.5  billion  is  allocated  to  the  Army,  about  A. 2 billion  to  the  Air 
Force,  about  billion  to  the  Navy  and  Marine  Corps,  and  about  0.8 
billion  to  other  defense  agencies.  The  growth  in  R&D  support  in 
the  Department  of  Defense  has  been  predicated  on  the  assumption 
that  R&D  has  a very  significant  impact  on  the  operational  capability 
of  military  forces.  That  assumption,  however,  has  been  coming  under 
increasing  scrutiny  by  policymakers  in  the  federal  establishment, 
and  there  is  a growing  pressure  to  increase  the  incidence  of  research 
product  use.  The  same  pressures  are  apparent  in  other  agencies 
as  well.  For  example,  beginning  in  FY  1977>  Research  Applied  to 
National  Needs  (RANN)  Division  of  the  National  Science  Foundation 
has  been  directed  by  the  Congress  to  establish  procedures  for  re- 
porting on  the  utilization  of  research  results.  These  kinds  of 
pressures  have  provided  a good  deal  of  the  impetus  for  a national 
conference  in  which  people  who  are  grappling  with  the  problem  of 
research  utilization  can  share  experiences,  needs,  frustrations  and 
successes,  and  can  together  formulate  tangible  policy  recommendations 
for  improving  the  utiMzation  of  R&D  products. 

The  challenge  is  indeed  at  hand.  We  are  aware  of  the  fine  work 
underway  in  each  of  the  military  services  and  in  other  federal 
agencies.  The  steering  committee  is  very  optimistic,  and  we  want 
you  to  know  we  intend  to  do  all  we  can  to  establish  effective 
channels  and  to  build  new  partnerships  that  would  increase  the  benefit 
to  be  derived  from  the  RDT&E  enterprise  now  and  in  the  future. 

With  that,  I'd  like  to  introduce  to  you  the  Commanding  Officer  of 
the  Naval  Personnel  Research  and  Development  Center  here  in  San 
Diego,  Captain  Jim  Clarkin. 
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WELCOMING  REMARKS  AND  INTRODUCTION  OF  KEYNOTE  SPEAKER 


CAFT  James  J.  Clark-in,  USN 
Cormanding  Officer 
Navy  Personnel  R&D  Center 


I had  planned  on  greeting  you  with  "Good  morning,  ladies  and  gentlemen," 
but  I note  with  some  concern  that  there  is  only  one  lady  present  in  the 
room,  and  I'm  wondering  if  in  some  obscure  fashion  this  might  be  related 
to  the  problems  of  utilization  of  personnel  R&D.  I do  welcome  you 
sincerely,  both  personally  and  on  behalf  of  the  Navy  Personnel  R&D 
Center,  and  I wish  you  every  success  in  your  work  during  the  next 
several  days. 

It  is  quite  proper  that  your  symposium  includes  not  only  representation 
from  each  of  the  military  services  and  the  Department  of  Defense,  but 
also  involves  participants  from  other  agencies,  from  academia,  and 
from  private  research  activities.  Under  the  best  circumstances,  an 
even  broader  participation  would  be  desirable,  for  it  is  not  only  the 
utilization  of  people-related  research  and  development  by  the  military 
services  but  also  the  behavioral  and  managerial  sciences  that  are.  under 
intensive  scrutiny  and  questioning  today. 

In  fact,  as  you  are  well  aware,  our  society  and  its  elected  arid 
appointed  officials  are  looking  skeptically  at  the  entire  spectrum  of 
technology.  During  the  past  two  decades,  a great  deal  of  attention 
and  concern  has  been  focussed  on  the  continually  increasing  degree 
of  specialization.  Much  of  the  concern  has  been  with  the  narrowness 
of  perspective  that  is  often  attendant  to,  or  a byproduct  of, 
specialization.  You  are  all  familiar  with  the  projections  that  if 
we  continue  at  our  present  pace,  by  the  year  x we  wi 1 1 have  more 
scientists  than  people  and  will  be  producing  more  technical  papers 
and  reports  than  can  be  stored  on  the  earth's  surface.  The  fact 
remains  that  we  have  continued  and  will  continue  to  specialize.  One 
cost  of  specialization  is  the  need  to  concern  ourselves  with  problems 
of  interdependence.  In  that  spirit,  we  need  to  give  increasing 
attention  to  our  dependence  upon  one  another  within  the  R&D  community 
and  to  our  interdependence  with  the  sponsoring  and  using  components 
of  the  larger  community  of  which  we  are  a part.  The  cost  of  this 
interdependence  is  measured  in  terms  of  time,  effort  and  intelligent 
compromise  and  adjustment.  We  will  have  to  accept  these  costs. 

In  welcoming  you  and  wishing  you  well  in  your  deliberations,  I would 
like  to  express  my  hope  that  you  will  aspire  to  make  the  symposium  a 
truly  productive  effort.  Most  of  you  are  already  aware  of  earlier 
delineations  of  problems  which  hinder  the  utilization  of  R&D 
efforts.  The  Ginnesburg  report,  and  more  recently  the  laboratory 
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I utilization  studies,  are  cases  in  which  many  of  the  utilization 

problems  were  quite  well  defined.  If  your  symposium  is  to  be 
productive,  as  I hope  it  will  be,  it  must  go  beyond  the  listing  of 
problems  or  constraints  to  utilization.  To  be  productive,  you  must 
, tackle  the  more  difficult  task  of  defining  the  steps  necessary  to 

’ remove,  or  at  least  reduce  in  degree,  existing  constraints  to  the 

utilization  of  people-related  R&O.  To  do  this,  I believe  you  must 
look  at  the  constraints  in  your  own  commun i ty--the  R&D  community-- 

as  well  as  those  to  be  found  in  the  external  world  of  sponsors  and 

users.  If  both  internal  and  external  constraints  are  recognized, 
then  the  prescriptions  for  removing  or  lessening  them  will  have  far 
^ greater  value. 

[ One  of  the  more  difficult  tasks  in  achieving  greater  utilization  of 

I R&D  products  may  be  in  making  intelligent  compromises,  and  in  making 

^ adaptations  or  changes  in  our  individual  and  collective  behavior 

f within  the  R&D  community.  Having  recognized  that,  I will  proceed 

f to  a change  in  my  own  behavior  and  address  the  more  important  task 

j I've  come  to  perform  here,  which  is  to  introduce  my  good  friend, 

I Admiral  Joe  Metcalf. 


KEYNOTE  SPEAKER 


EADM  Joseph  Metaalf  III,  USN 
Assistant  Chief  of  Naval  Personnel 
and  Management  Information 


Thanks,  Jim.  Since  Captain  Clarkin  has  already  given  my  speech,  I may 
' as  well  sit  down  right  now.  Seriously,  I think  that  Jim  has  given  you 

I an  excellent  picture  of  what  the  challenge  is  all  about. 

I would  like  to  compliment  Captain  Clarkin  and  NPRDC  for  being  brave 
enough  to  host  this  symposium.  The  subject  you  propose  to  address 
is  very  difficult.  There  is  hardly  a problem  I can  think  of  that  is 
more  pressing,  more  urgent,  and  more  fundamental  to  the  business  in 
which  you  are  professionals.  As  Captain  Clarkin  mentioned,  the 
area  of  R6D  utilization  is  of  great  concern  to  me.  When  I sit  down 
with  my  RSD  colleagues,  we  usually  get  off  on  the  subject  of  how  to 
I deal  with  this  problem  of  utilization.  It  is  tremendously  difficult 

' and  always  frustrating. 

I would  like  to  characterize  my  remarks  as  bridge  building,  a term 
I borrowed  from  Mr.  Malehorn's  provocative  paper,  because  what  we 
i are  really  doing  in  research  utilization  is  building  a bridge 

I between  the  conduct  of  research  and  its  implementation.  Bridge 

1 building  involves  spanning  gaps,  but  the  gaps  in  themselves  are 

I 
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not  the  important  thing.  We  know  the  nature  of  the  gaps  in  the  RSD 
business--fai lure  to  communicate,  and  lack  of  understanding,  among 
other  things.  The  more  important  task  before  you  is  to  define  the 
ends  of  the  bridge,  the  abutments,  because  in  my  judgment  that  is 
where  both  the  problems  and  the  solutions  lie. 

In  the  RSD  business,  there  are  three  islands  to  bridge  and  join: 

(1)  the  consumers,  who  utilize  the  products  of  RSD;  (2)  the  highly 
talented,  dedicated  individuals  who  conduct  research;  and  (3)  the 
managers  of  research,  who  provide  the  lubricant  (and  sometimes  the 
lack  of  lubricant)  between  the  consumer  and  the  researcher.  It  is 
this  third  group,  the  research  managers,  who  in  my  view  are  probably 
the  key  to  the  problem  of  utilization.  They  pay  the  bills,  serve 
as  coordinators,  and  have  a very  important  review  function.  In  short, 
the  critical  element  in  the  equation  of  bridge  building  is  the 
professional  in  RsD  management. 

If  I were  to  survey  the  audience  today,  I believe  I would  probably 
find  that  the  researchers  and  the  managers  of  research  far  outnumber 
the  users,  whom  I represent.  This  is  a chronic  problem.  In  fact, 

I'm  afraid  there  may  even  be  a diminishing  community  of  interested 
users.  If  we  can't  get  more  consumers  actively  involved  in  solving 
the  utilization  problem  I think  the  task  that  you  people  have  set 
for  yourselves  is  going  to  be  very,  very  difficult.  In  other  words, 

I would  suggest  that  the  initial  problem  is  how  to  get  more  of  us 
users  actively  involved  in  the  busines  of  R6D  and  its  proper 
utilization.  One  end  of  the  bridge  seems  to  be  lost  in  the  fog. 

To  illustrate  why  we  find  this  attitude  of  apparent  indifference  by 
users,  I would  like  to  borrow  a quotation  (from  Forbes  magazine)  which 
relates  to  the  options  market.  "Potential  i nvestors Tin  the  options 
market)  would  do  well  to  remember  that  options  have  generally  proved 
to  be  better  business  for  brokers  than  they  have  for  investors."  Many 
consumers  of  R&D  would  rephrase  that  quotation  in  the  following  way: 
"Potential  users  of  R&D  would  do  well  to  remember  that  research 
has  generally  proved  to  be  better  business  for  researchers  and 
research  managers  than  for  the  consumer."  Now,  this  perception  is 
one  of  the  gaps  that  I think  you  must  address  in  your  bridge  building. 

It  is  a perception  held  by  many  of  us;  and  whether  we  like  it  or  • 
not,  it  is  a fundamental  block  in  the  proper  commissioning  and 
utilization  of  much  R&D  and  in  particular,  of  human  resources  research. 

This  feeling  that  research  very  often  is  done  for  the  sake  of 
research  is  just  a symptom  of  the  utilization  malaise,  it  is  not  the 
heart  of  the  problem.  I believe  the  basic  disease  is  that  neither 
the  consumer,  the  manager,  nor  the  human  resources  researcher 
consistently  or  effectively  plans  for  implementation.  Thus,  your 
prime  task  at  this  symposium  will  be  to  explore  ways  to  inject  the 
discipline  of  planning  into  the  process  of  testing  and  implementing 
promising  R&D  end  products.  The  key  to  the  bridge-building  business 
is  to  plan  the  bridges. 
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1 would  like  to  explore  three  problem  areas  that  I feel  contribute  to 
the  lack  of  planning  for  utilization:  (l)  resistance  to  change, 

(2)  conflict  of  interest  between  the  producers,  the  managers,  and 
the  consumers  of  R&D,  and  (3)  the  difficulty  in  developing  reasonable 
criteria  of  success. 

First,  let  me  comment  on  resistance  to  change.  I am  not  going  to 
offer  illustrations;  we  each  have  our  own  horrible  examples.  I 
would  like  to  observe,  however,  that  we  all  tend  to  look  at  ourselves 
as  absolute  paragons  of  flexibility.  Thus,  when  the  subject  is 
resistance  to  change,  you  will  generally  find  that  fingers  are 
pointed  in  all  directions  but  inward.  As  you  go  about  your  labors 
in  this  symposium,  I suggest  that  you  take  a healthy  look  at 
yourselves  and  the  institutions  you  represent,  and  see  if  you  can 
knock  out  any  cobwebs.  Something  is  going  to  have  to  change  in  the 
way  we  do  business  or  we  are  not  going  to  be  successful.  You  might 
as  well  not  be  here  if  you  do  not  produce  a program  of  recommended 
change  in  the  way  we  plan  RsD  utilization. 

The  second  area,  conflict  of  interest  between  R&D  managers,  producers, 
and  consumers  is  perhaps  the  best  understood.  Left  to  your  own 
devices,  1 suspect  you  would  concentrate  your  efforts  here.  In  my 
view,  however,  it  is  not  the  key  area.  There  are  conflicts  of 
interest.  For  example,  the  researcher  tends  to  focus  his  attention 
on  methodology,  whereas  the  consumer  is  more  interested  in  operational 
responsiveness..  The  researcher  (particularly  if  engaged  in  basic 
research)  generally  is  not  interested  in  implementation  planning,  and 
perhaps  does  not  even  know  how  to  do  it.  At  the  same  time,  the 
manager  of  research  sits  in  the  middle  and  is  concerned  with  balancing 
programs,  putting  things  in  the  proper  categories  (6.1,  6.2,  etc.), 
showing  that  the  books  are  in  order,  defending  programs  before 
Congress,  and  making  sure  they  all  make  sense.  Unfortunately, 
these  interests  are  very  often  in  fundamental  conflict. 

Let  me  go  back  for  a moment  to  the  problem  of  resistance  to  change, 
which  is  founded  in  conflict  of  interest.  In  many  instances  the 
consumer  is  afraid  of  the  extended  impact  of  R6D,  of  changes  that 
vie  or  our  bureaucracies  are  not  willing  to  accept.  Automation  is 
a case  in  point;  here  the  implementation  of  a single  change  may 
imply  revolution  in  the  way  we  do  business.  We  as  consumers  are 
very  often  unsure  about  the  implications  of  given  changes,  and 
thus  we're  afraid  to  take  them  on.  At  least  in  part,  the  consumer's 
reluctance  is  based  on  the  fact  that  he  feels  that  he  is  more 
sensitive  to  the  political  impact  of  human  resources  R&D  than  is  the 
researcher,  because  he  has  to  deal  with  Congr>*ss,  for  example, 
and  has  to  take  the  flak  when  something  goes  ..rong. 


Now,  let's  take  a look  at  one  particularly  troublesome  problem  of 
conflict  of  interest:  the  clash  of  the  time  differential.  With 
respect  to  a given  R&D  activity,  users  are  generally  in  the  picture 
for  very  short  periods  of  time.  In  contrast,  the  R&D  community 
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is  typically  involved  for  much  longer  periods  of  time.  It  always 
requires  more  time  to  develop  a solution  than  to  pose  a problem.  Thus, 
very  often  the  consumer  who  poses  the  problem  and  asks  for  the  re- 
search to  be  done  is  gone  when  the  solution  arrives. 

Further,  the  researcher  may  have  little  interest  in  a problem  that 
requires  the  application  of  yesterday's  technology.  Also,  researchers 
contend  that  consumers  of  R£D  often  want  some  kind  of  magic  pill  to 
provide  a quick  fix  and,  thus,  open  themselves  to  old  snake  oil 
remedies.  This  expectation  nurtures  many  of  the  so-called  "beltway 
bandits"  that  surround  Washington.  We  get  for  our  money  what  we 
ask  for--a  scientific  gloss  on  any  subject  to  support  whatever 
conclusion  the  consumer  demands.  The  magic  pill  syndrome  is  often 
characterized  by  the  consumer  asking  a question  for  which  there  is 
no  reasonable  or  immediate  answer.  We  tend  to  turn  to  R£D  for  the 
solution  to  the  Gordian  Knot.  When  we  receive  less  than  a satis- 
factory answer  to  an  impossible  problem  we  carve  one  more  notch 
on  the  stick  with  which  we  flagellate  the  research  community.  The 
problem  is  that  many  of  the  consumers  of  RSD  do  not  understand  the 
fact  that  if  the  answers  they  seek  were  simple,  they  wouldn't  be 
going  to  the  research  community  to  get  them. 

Another  aspect  of  the  conflict  of  interest  is  that  the  consumer 
generally  prefers  minimum  risk  while  the  researcher  thrives  on 
risk.  In  fact,  risk  is  what  research  is  all  about.  Studies  of  the 
behavior  of  managers  in  industry  have  shown  that  they  do  not  try 
to  maximize  profit,  because  that  generally  entails  maximum  risk. 

The  max  profit  notion  is  rhetorical  fantasy.  Managers  maximize 
stability  rather  than  risk  upsetting  the  organization.  This  is 
also  true  in  the  community  of  military  consumers  of  which  I am 
a member,  and  in  my  experience,  is  also  present  to  a high  degree 
amongst  the  middlemen--the  research  managers.  We  often  talk  of 
change,  but  we  seldom  behave  in  a way  that  promotes  it. 

Perhaps  the  most  fundamental  conflict  between  producers  and  consumers 
lies  in  the  issue,  "Are  we  studying  the  right  problem?"  Last  week 
(following  a presentation  by  Dr.  Hackman  of  Yale  on  a study  of  job 
motivation),  the  Chief  of  Naval  Personnel  lamented  about  this  to 
a group  of  researchers  and  senior  managers.  From  Admiral  Watkin's 
view,  it  was  a splendid  piece  of  work,  but  was  sub-optimization. 

What  was  really  needed,  he  said  was  research  on  the  process  of 
change  associated  with  an  all-volunteer  force.  Dr.  Hackman  was 
focusing  on  a way  to  increase  motivation;  Admiral  Watkins  was 
looking  at  the  bigger  problem.  He  said,  "Here  we  have  a situation 
where  our  forces  are  predominantly  volunteers.  They  should  be 
happier,  but  they  are  less  happy.  Desertion  rates  are  up,  and 
reenlistment  rates  are  not  substantially  different  than  they  were 
during  the  days  of  the  draft."  Dr.  Hackman  made  the  observation 
that  we  have  mountains  of  data  that  indicate  that  the  rate  of 
attrition  of  the  nonhigh  school  graduate  is  twice  that  of  his 
high  school  graduate  peer.  The  solution  suggested  by  intuition 


and  the  research  community  seems  to  be  to  recruit  high  school 
graduates.  Admiral  Watkins  pointed  out  that  a better  solution  might 
be  to  attempt  to  change  the  propensity  of  nonhigh  school  graduates 
to  attrite,  since  not  enough  high  school  graduates  choose  to  volunteer. 
The  basic  issue  is  the  relevance  of  the  research.  I suspect  that 
Dr.  Rostker  will  deal  with  this  in  some  detail  tonight. 

I surmise  that  many  of  the  answers  to  the  questions  that  Admiral 
Watkins  proposed  to  Dr.  Hackman  and  to  those  of  us  who  were  present 
are  available  on  the  shelves  somewhere  in  your  research  communities. 

The  problem  is  both  one  of  getting  access  to  this  information  and 
of  using  it. 

The  third  problem  area  contributing  to  the  lack  of  planning  for  utili- 
zation of  R&D  is  the  difficulty  in  developing  reasonable  criteria 
for  success — so  that  we  can  assign  accountability.  I recognize 
that  after  25  years  in  the  military  the  term  accountability  probably 
has  a very  different  connotation  to  me  than  to  you--one  that  is 
hard,  stark,  black  and  white.  That  connotation  undoubtedly  cannot 
be  applied  to  the  business  of  research,  nor  would  it  ensure  the 
utility  of  research.  So  the  problem  is,  how  do  we  measure  the 
utility  of  research,  and  who  is  accountable  for  starting  research 
that  is  poorly  defined  with  reference  to  the  problem  at  hand. 

Perhaps  one  test  of  accountability  is  survival.  And  if  survival 
isn't  a satisfactory  criterion,  perhaps  we  should  look  at  success-- 
if  only  we  could  measure  it.  The  idea  I'm  trying  to  get  at  is  that 
it  is  very  difficult  to  formulate  criteria  (relevance,  utility, 
viability,  etc.)  for  identifying  and  promoting  good  research. 

In  my  experience,  worthwhile  "people  research"  has  multiple  dimensions 
and  uncertainties.  Because  bureaucratic  organizations  do  not  like 
multiple  measures,  we  try  to  collapse  them  into  some  sort  of  simpler 
index  or  "bottom  line"  denoting  good  or  bad.  Mark  Twain  observed 
that  there  is  a simple  solution  for  every  complex  problem  and  it  is 
always  wrong.  And  I think  that  observation  is  particularly  apt 
in  the  business  of  people  research,  especially  with  reference  to 
implementation.  But  that  does  not  mean  there  is  any  less  need  for 
accountabi 1 i ty . 

I would  suggest,  then,  that  an  implementation  plan  is  probably  as 
good  a surrogate  of  accountability  as  we  are  going  to  find.  In 
other  words,  if  we  have  a well  thought  out  plan,  we  can  judge  how 
well  we  carry  out  that  plan. 

Perhaps  the  most  reasonable  criterion  of  success  is  whether  the  re- 
search was  or  is  usable.  Far  too  much  research  on  human  behavior, 
organizational  theory,  and  leadership  explains  history  very  well, 
but  too  often  does  not  adequately  explain  it  in  terms  of  factors 
that  management  can  control  or  use.  The  interesting  question, 
often  left  unanswered,  is  "what  can  the  consumer  do  with  the 
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research  result,  given  that  he  accepts  what  the  researcher  says 
as  true?"  Even  where  the  variables  are  controllable  by  management, 
the  researcher  usually  provides  no  estimate  of  what  the  impact  of 
future  management  decisions  and  change  will  be.  The  consumer  is 
left  to  answer  the  question  of  what  it  will  cost  to  effect  the 
change  suggested  by  the  researcher,  and  what  the  most  likely  outcome 
will  be.  Again,  if  we  were  required  to  anticipate  the  answers  to 
some  of  these  questions  we  might  be  able  to  bridge  the  gap  between 
the  consumer  and  the  user.  For  example,  it  has  been  shown  that 
pay  grade  is  related  to  first-term  retention.  If  I,  as  a manager, 
made  everyone  an  E-5,  would  they  reenlist?  Considering  cost,  what 
are  the  diminishing  returns  for  such  a suggestion?  In  short,  too 
little  R&D  has  criteria  applied  by  which  either  the  researcher, 
the  research  manager,  or  the  consumer  can  estimate  the  risk 
associated  with  the  implementation  of  the  research. 

We  have  covered  three  points:  resistance  to  change,  conflict  of 
interest,  and  the  lack  of  criteria.  I think  that  these  points 
adequately  identify  the  malaise  that  we  have  before  us.  The  problem 
is  summarized  nicely  in  Dr.  Drucker's  paper,  in  which  he  says  that 
too  often  we  have  experienced  the  frustration  of  trying  to  promote 
utilization  of  R£D  products  when  virtually  no  money  has  been  allocated 
for  that  purpose,  few  personnel  can  be  legitimately  assigned  to 
implementation  activities,  and  there  is  little  or  no  responsibility 
in  user  commands  for  seeing  that  the  product  gets  used.  Products 
have  been  known  to  get  into  the  correct  systems,  nonetheless,  but 
generally  through  ad  hoc  arrangements  spurred  by  someone's  conscience 
and  with  funding  from  God  knows  where.  In  short,  no  up-front 
planning. 

It  is  clear  to  me  that  we  cannot  continue  on  this  road.  I suggest  the 
following  points  for  your  consideration  in  this  conference.  I would 
say  that  point  one  is  to  develop  a means  to  plan  an  organized  way  of 
requiring  that  the  potential  use  of  a given  piece  of  research, 
even  6.1  research,  is  made  explicit.  Point  two  is  to  do  some  more 
planning;  point  three  is  to  plan  some  more.  Fundamentally,  we 
must  get  into  the  business  of  developing  R&D  utilization  plans.  Through 
planning  we  must  educate  users  and  consumers  so  that  they  will  have 
a better  understanding  of  the  power  of  R&D,  of  its  limitations,  and 
of  what  can  be  expected  of  R£D,  particularly  in  specified  time  frames. 
Going  one  step  further,  I suggest  that  it  is  the  principal  re- 
sponsibility of  the  R&D  manager  to  articulate  and  carry  out  this 
planning.  It  also  is  the  responsibility  of  the  individual  research 
project  staff  to  participate  in  the  planning. 

I think  we  need  to  develop  better  methods  to  categorize  R&D,  particu- 
larly human  resources  research.  For  example,  the  program  categories 
of  6.1  and  6.2,  etc.,  were  designed  to  produce  "rat  screws"  (i.e., 
hardware)  and  are  generally  inappropriate  to  the  human  resources  area. 

We  also  need  to  revise  the  way  R&D  programs  are  evaluated.  Presently, 
these  programs  are  generally  examined  in  much  more  detail  by  the 
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manager  of  RSD  than  by  the  consumer.  For  example,  at  a recent 
program  review  conducted  at  NPRDC , which  was  very  professionally 
done  from  the  perspective  of  the  RSD  manager,  not  a single  consumer 
was  present.  Something  must  change  in  this  area.  Perhaps  we  need  to 
set  up  a bureaucratic  procedure  that  assigns  an  obligation  to  the 
consumer,  as  part  of  his  accountability,  to  evaluate  the  RSD  program. 

We  also  need  to  reduce  the  fragmentation  of  RSD  management.  There 
are  many  research  programs  in  which  there  appear  to  be  more  managers 
than  researchers. 

I would  suggest  that  the  idea  of  resource  programming  and  research 
implementation  go  hand  in  hand.  If  you  are  going  to  put  a program 
into  effect  these  days,  you  must  allocate  resources  three  years  in 
advance.  Clearly,  this  means  that  in  order  to  put  research  into 
effect,  you  must  plan  well  in  advance  and  anticipate  the  funds  that 
will  be  required  if  the  research  is  successful  enough  to  warrant 
uti 1 ization. 

Formal  assignment  of  responsibility  for  the  planning  and  implementation 
of  an  RSD  product  must  be  made  explicit.  I am  generally  sympathetic 
to  the  viewpoint  of  the  researcher,  who  produces  the  product  and 
feels  that  he  is  not  responsible  for  selling  it.  In  fact,  I think 
that  some  researchers  would  be  better  kept  in  the  back  room, 
while  others  should  be  trooped  out  into  public  view.  Good  scientists 
are  often  poor  at  articulating  the  concepts  of  utilization.  They 
often  have  a different  set  of  values  and  goals,  which  1 pointed 
out  earlier  in  discussing  conflict  of  interest.  The  scientist 
exploring  the  unknown  may  not  be  interested  in  programming  and 
implementing  the  results  of  his  research.  I am  not  sure  that  he 
should  be,  but  someone  has  to  be  responsible  for  understanding  what 
is  required  not  only  to  get  the  research  accomplished  but  to  follow 
it  through.  In  my  judgment,  the  real  responsibility  for  this  aspect 
of  R&D  lies  with  the  consumers  and  the  R6D  management  professionals. 

In  conclusion,  I would  like  to  reemphasize  that  I believe  planning 
is  the  most  essential  element  in  this  business  of  building  bridges 
between  consumers,  producers  and  managers  in  order  to  promote 
better  utilization  of  R&D.  I wish  you  the  best  of  luck  in  your 
explorations  into  this  swamp  of  human  resources  R&D  utilization. 

It's  a tough,  murky  problem.  If  it  were  not,  you  would  not  be  here. 

As  Captain  Clarkin  said,  it  probably  is  one  of  the  most  urgent  tasks 
confronting  us  today.  Your  challenge  is  to  map  a strategy  for 
change,  one  which  will  be  more  likely  than  the  present  modus 
operand!  to  bring  about  timely  utilization  of  promising  R&D  results. 

As  I have  already  stated,  if  we  are  going  to  improve  R&D  utilization, 
we  must  improve  accountability  for  utilization,  and  with  today's 
austere  funding  climate,  the  reward  for  a particular  RSD  project 
may  very  well  be  its  survival.  However,  this  is  not  the  real  objective. 
Remember  that  the  successful  utilization  of  an  effective  new  development 
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by  the  operational  community  may  mean  the  difference  between  victory 
and  defeat.  The  basic  task  that  you  have  is  to  get  the  research 
horse  connected  up  with  the  operational  wagon.  It  is  not  going  to 
be  easy. 

Now  I would  like  to  introduce  Colonel  Hank  Taylor  who,  in  some  measure, 
I've  been  flagellating  here  today.  But  he  is  one  of  the  most  competent 
people  in  this  business. 


RDT&E  UTILIZATION:  A PERSPECTIVE  FROM  THE  OFFICE  OF  DDR&E 


COL  H.  L.  Taytov,  USAF 

Military  Assistant  for  Training  and  Personnel  Teahnology 
Office  of  the  Secretary  of  Defense 


I am  pleased  to  have  the  opportunity  to  speak  at  this  important 
symposium.  My  topic  is  RDT&E  utilization:  a perspective  from  the 
Office  of  DDR&E.  I'd  like  at  the  outset  to  thank  Dr.  Sands  for 
allowing  me  30  minutes  to  cover  this  topic,  because  if  I had  had  a 
shorter  time,  with  the  reading  of  the  title  of  the  symposium  and 
the  title  of  my  presentation  I think  1 would  have  been  out  of 
business  before  I even  got  started. 

I think  Admiral  Metcalf  put  his  finger  on  many  of  the  problems  that 
our  community  faces,  and  I want  to  emphasize  "our"  community,  because 
utilization  requires  a joint  effort  between  the  researcher  and  the 
user.  I think  he  not  only  pointed  to  the  deficiencies  in  the  way 
we  interact  with  each  other  but  that  he  also  probably  flailed  himself 
as  much  as  he  flailed  any  of  the  R&D  users.  I think  we  need  to  focus 
on  this  problem  of  the  interfaces  between  the  user  and  the  researcher 
and  the  R&D  manager. 

As  both  an  advocate  and  critical  reviewer  of  the  Department  of  Defense 
Research  and  Technology  programs  in  people-related  research,  it  is 
probably  the  understatement  of  the  year  to  say  that  I am  interested 
in  utilization.  We  have  to  be  concerned  with  research  utilization, 
because,  as  Admiral  Metcalf  so  clearly  indicated,  from  an  R&D 
management  standpoint  it  typically  indicates  whether  your  program 
will  continue  to  survive  or  not.  On  a broader  perspective,  R&D 
utilization  has  the  potential  of  impacting  on  combat  readiness  and 
on  national  security. 

Certain  external  factors  that  impinge  on  the  area  of  utilization  of 
people-related  research  should  be  reviewed  and  considered  in  your 
deliberations.  This  area  has  probably  had  more  review  per  dollar 
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than  any  other  RED  program  around,  both  within  the  military  services, 
the  Department  of  Defense,  and  the  Congress.  It  is  an  area  that  is 
not  clearly  articulated  through  any  chain  of  command  in  terms  of  the 
potential  impact.  We  don't  market  our  research  very  well.  We  don't 
provide  clear  understandings  to  the  people  who  are  trying  to  manage 
the  R&D  or  to  apply  the  results.  Perhaps  you  can  come  up  with 
suggestions  for  overcoming  these  deficits. 

Over  the  past  decade  we've  put  a great  deal  of  emphasis  on  utili- 
zation within  the  Department  of  Defense.  For  those  of  you  that 
have  been  in  R&D  management  or  associated  with  RED  for  the  last 
decade,  you  will  remember  that  in  the  late  60's  and  early  70's 
the  Congress  showed  an  intense  interest  in  relevancy  of  research. 

I submit  to  you  that  the  behavioral  and  social  sciences  RED,  with 
its  emphasis  during  this  time  period  on  foreign  affairs  research,  was 
one  of  the  principal  programs  that  created  this  Congressional 
concern  about  relevancy,  a concern  that  spread  to  all  of  DoD's 
research  and  development.  You  are  probably  familiar  with  the 
Mansfield  Amendment  to  an  early  1970  appropriations  bill,  which 
stated  that  all  defense  research  and  development  must  have  a 
direct  and  apparent  relationship  to  the  DoD  mission.  What  consti- 
tutes a direct  relationship  is  not  easy  to  define,  but  the  emphasis 
was  to  the  overall  DoD  mission,  either  in  the  development  of  hardware, 
weapon  systems  or  subsystems,  or  in  the  maintenance  of  combat  readi- 
ness through  better  training,  better  personnel  management,  or 
better  manpower  management. 

The  stipulation  that  the  research  must  have  an  apparent  relationship 
to  the  DoD  mission  was  even  less  specific.  I was  at  the  Air  Force 
Systems  Command  at  that  time,  when  a review  of  all  the  RED  programs 
in  the  Air  Force  was  conducted  as  a result  of  the  Mansfield 
Amendment.  We  struggled  with  the  problem  of  how  one  could  determine 
whether  a particular  research  effort  had  a direct  and  apparent 
relationship  to  the  DoD  mission  and  we  finally  decided  that  the 
best  criterion  that  we  could  come  up  with  was  to  let  the  night 
janitor  read  the  project,  and  if  he  could  see  the  apparent  relation- 
ship then  we  were  probably  on  safe  ground. 

In  a more  serious  vein,  there  was  a very  critical  and  very  serious 
review  of  all  defense  research  and  development  during  that  time.  In 
the  Systems  Command,  we  sorted  all  projects  into  three  categories 
based  on  the  review  of  the  project  write-up.  One  contained  those 
projects  where  the  relationship  was  both  direct  and  apparent. 

Another  category  contained  those  projects  in  which  there  appeared  to 
be  a relationship  but  it  was  not  clearly  articulated  in  DD-1498, 
the  project  write-up.  (Each  of  those  in  this  second  category  had 
to  be  rewritten  or  the  project  would  not  be  continued.)  The  third 
group  contained  those  projects  in  which  the  relationship  was  neither 
direct  nor  apparent.  These  were  cancelled.  Each  of  the  military 
services  went  through  this  exercise  for  all  RED.  The  result  was 
a considerable  restructuring  of  the  programs  from  the  standpoint 
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of  both  relevancy  and  emphasis.  I think  this  was  the  start  of  the 
emphasis  that  we  are  seeing  today  in  the  Congress. 

Shortly  after  this  period,  behavioral  and  social  sciences  RCD  dipped 
to  a low  funding  point  of  somewhere  around  $37  million  in  technology- 
related  items.  This  was  about  the  time  of  the  Ginnesburg  Management 
Report,  which  recommended  an  increase  in  RDT&E  in  this  area,  along 
with  better  management  of  it.  I f we  look  at  the  trends  in  funding,  we 
see  that  RDT&E  in  this  area  has  increased  every  year  since  that  time. 
In  fact,  it  has  increased  far  out  of  proportion  to  the  rest  of 
RDT&E  funding.  However,  it  is  still  a struggle  both  to  defend 
the  programs  and  to  get  them  implemented  if  they  are  supported  and 
completed . 

The  next  major  historical  point  was  the  House  Appropr iat ions 
Committee's  review  and  report  on  the  1976  fiscal  budget  request.  The 
Committee  recommended  a 50  percent  reduction  (from  $A0  million  to 
$20  million)  in  five  critical  program  areas  within  what  we  then 
called  human  resources  R&D.  The  recommendation  came  near  the  end  of 
the  first  quarter  of  the  fiscal  year.  If  implemented,  it  would  have 
had  a very  significant  effect  on  the  RSD  community.  In  light  of 
all  of  the  personnel  actions  that  would  have  been  required  to  meet 
the  reduced  budget,  we  simply  would  not  have  had  enough  money  left 
even  to  close  some  of  our  laboratories,  much  less  keep  them  open. 

There  were  also  some  very  critical  contractual  efforts  that  would 
have  had  to  be  terminated  before  their  completion. 

The  rationale  for  this  reduction  by  the  House  Appropriations  Committee 
was  their  concern  over  the  extent  of  utilization  of  research  in  this 
area.  We  were  successful  in  getting  a portion  of  these  cuts  restored; 
the  Senate  restored  half  of  the  $20  million  reduction  and  the  House 
acceded  to  the  Senate's  position.  You  might  say  that  people-related 
R&D  was  spared,  but  the  Congress  really  got  our  attention.  Since 
that  time,  there  has  been  a concerted  effort  within  the  R&D  management 
community  and  within  certain  segments  of  the  user  community  to  make 
the  utilization  of  people-related  RsD  clear.  There  also  have  been 
internal  programs  designed  to  look  at  utilization,  which  I'll  talk 
about  later. 

In  response  to  the  action  of  the  Congress,  we  restructured  the  human 
resources  RSD  program  so  that  we  could  better  see  its  various  parts. 

We  tried  to  make  the  program  responsive  to  the  Congressional  concerns 
and  still  maintain  a viable  effort  to  address  critical  DoD  needs. 

In  November  1975,  following  the  final  authorization  of  the  FY  76 
budget,  the  chairman  of  the  House  Appropriations  Committee  requested 
that  the  General  Accounting  Office  make  a detailed  review  of  these 
programs  so  that  a more  thorough  analysis  of  the  FY  77  defense 
budget  request  could  be  completed.  This  review  started  immediately 
and  continued  for  almost  a year  and  a half.  It  culminated  in  the 
GAO  report  published  April  22,  1977,  entitled  "Human  Resources 
Research  and  Development  Results  Can  Be  Better  Managed."  I would  like 
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to  take  a few  minutes  to  give  you  some  of  the  highlights  of  this 
report,  which  can  be  obtained  from  the  General  Accounting  Office. 

I will  give  enough  details  so  that  you  can  consider  this  report 
in  your  deliberations,  since  I think  it  is  a very  important  mile- 
stone. 

The  GAO  conducted  this  evaluation  over  a period  of  about  l8  months. 
They  developed  an  audit  trail  of  human  resources  research,  using 
the  technical  report  as  a basic  source  to  audit.  They  visited 
eight  organizations  within  the  Department  of  Defense  and  identified 
37^  reports  that  had  results  which  the  research  community  deemed 
ready  for  immediate  use.  By  the  GAO  definition  this  included  all 
research  intended  to  support  changes  in  regulations,  orders, 
doctrines,  policies  or  manuals;  research  intended  to  support 
changes  in  programs  of  instruction  or  training  programs;  and  re- 
search intended  to  support  changes  in  equipment. 

To  complete  the  audit  trail,  the  GAO  contacted  users  both  by  survey 
and  personal  interviews  to  determine  which  R&D  results  were  used 
and  which  were  not  used.  They  found  that,  by  their  criteria,  56 
percent  of  the  results  had  been  used,  38  percent  had  not  been  used, 
and  6 percent  were  being  considered  for  possible  use.  The  emphasis 
here  (and  in  our  opinion,  overemphasis)  was  clearly  on  immediate  use. 
We  also  feel  that  the  definition  of  use  is  quite  restrictive. 

However,  some  of  the  reasons  they  found  for  nonuse  are  very  in- 
structive. First,  many  of  the  users  believed  that  the  reported 
results  were  intended  for  information  only.  Second,  the  supposed 
users  (that  is,  people  who  were  identified  by  the  R6D  community  as 
potential  users  of  the  research)  had  not  seen  the  reports  or  were 
unaware  of  the  reported  results.  Third,  the  results  were  questioned 
or  were  believed  to  be  unusable  by  the  research  community. 

There  were  other  categories  in  which  the  results  were  clearly  used 
in  decision-making  but  which  were  considered  unused  by  the  GAO. 

In  our  estimation  these  situations  constituted  a legitimate  use 
of  defense  research  since  the  results  did  have  impact  on  decision- 
making. However,  counting  them  doesn't  substantially  change  the 
percentages.  (it  would  put  the  none-use  category  down  to  about  25 
percent  rather  than  38  percent.) 

What  we  need  to  focus  on  are  those  instances  where  the  user  cannot 
determine  from  the  research  report  that  it  is  intended  to  do  more 
than  simply  increase  his  state  of  knowledge,  or  where  he  doesn't 
even  receive  the  report.  Some  of  these  things  can  be  handled  by 
management  action,  but  I think  it's  clear  that  there  is  ample 
opportunity  to  effect  change  within  the  management  of  human  re- 
sources R&D. 

As  a result  of  this  study,  the  GAO  made  four  recommendations  to  DoD 
which  are  intended  to  improve  utilization  management.  The  first  one 
was  that  criteria  need  to  be  developed  to  identify  research  and 
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development  results  with  immediate  use  potential.  They  concluded 
that  without  such  criteria,  results  cannot  be  identified  and  tracked, 
and  related  implementation  problems  cannot  be  identified. 


We  have  reviewed  this  recommendation  and,  in  general,  support  the 
GAO's  conclusion.  Criteria  are  needed  to  identify  research  efforts 
that  have  potential  use,  and  to  identify  intended  users  and  the  ways 
in  which  the  results  may  be  used.  We  have  not  responded  to  this 
recommendation  yet,  but  we  have  drafted  a response.  Although  I am 
probably  speaking  out  of  school,  1 would  at  least  like  to  give  you 
an  idea  of  what  the  Department  of  Defense  is  considering  in  terms 
of  a response,  because  I think  this  is  very  important  and  pertinent 
to  your  actions  here. 


As  you  recall,  the  GAO  used  three  criteria  for  judging  the  use  potential 
of  a given  research  project  (i.e.,  research  intended  to  support  (l ) 
change  in  education  or  training,  (2)  change  in  equipment,  or  (3)  change 
in  regulations,  orders,  policy,  doctrine  or  manuals).  The  GAO 
recommended  that  DoD  use  these  three  definitions  of  use  as  a basis 
for  developing  criteria  to  determine  immediate  use  potential.  We 
accept  this  but  think  that  the  criteria  are  too  narrow,  and  we  are 
looking  at  additional  criteria  that  should  be  used  to  identify 
research  with  immediate  use  Dotential.  Our  intention  at  the  present 
time  is  to  require  that  each  published  technical  report  in  the  human 
resources  area  provide  a statement  concerning  the  intended  user  or 
user  community  and  the  suggested  use  of  any  product  or  findings.  This 
will  include  those  research  reports  that  essentially  feed  other  R6D 
efforts,  since  the  R6D  community  is  a consumer  of  its  own  RDT6E. 

In  short,  we  are  looking  at  it  from  a very  broad  perspective,  and 
intend  to  include  those  projects  in  which  the  research  is  designed 
to  advance  the  state  of  knowledge  and  to  provide  additional  in- 
formation rather  than  change  equipment,  programs,  or  policy.  We 
believe  that  the  responsibility  for  this  lies  with  R&D  management. 


The  second  conclusion  in  the  GAO  report  was  that  communication  between 
researchers  and  users  must  be  improved.  This  is  a motherhood 
statement,  but  I think  it  is  also  valid.  It  is  evident  that  better 
communication  will  serve  to  identify  problems  and  provide  more  timely 
solutions.  This  goes  back  to  Admiral  Metcalf's  conclusion  that 
better  planning  in  all  phases  of  defense  RDT&E  is  one  of  the  things 
that  we  need  to  focus  on.  Obviously,  we  have  deficiencies  in  our 
communications  process.  I think  we  all  are  aware  of  some  of  the 
problems . 


In  my  opinion,  we  are  actively  working  to  develop  both  formal  and 
Informal  solutions  to  these  very  difficult  problems.  One  idea  is 
to  institute  a periodic  utilization  report;  the  Navy  is  actively 
pursuing  this  effort.  The  Air  Force  and  Army  both  have  changed 
their  regulations  regarding  human  resources  research  in  the  last 
year  or  two  in  a way  that  specifically  identifies  the  need  for  user 
involvement  in  the  R&D  process,  from  the  requirements  stage  all  the 
way  to  the  transition  of  R&D  to  the  user,  and  also  identifies  the 
points  for  interface  during  the  R&D  process. 
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We  have  tried  several  other  things  over  the  years.  One  is  having 
field  detachments  directly  collocated  with  users.  In  many  cases, 
this  helps  the  communications  process  both  on  the  requirements  end 
and  on  the  utilization  end.  There  are  examples  of  this  in  all 
three  military  departments.  Another  strategy  is  holding  conferences, 
such  as  this  one  that  NPRDC  is  hosting,  to  try  to  come  to  grips  with 
the  communications  problem.  Another  technique  that  is  being  tried 
by  some  of  the  military  departments  is  designating  representatives  at 
intermediate  command  levels  to  serve  as  intermediaries  between  the 
research  and  user  communities.  Although  we  are  doing  a lot  in  the 
area  of  user/researcher  communication,  I would  submit  to  you  that 
it  needs  further  work. 

The  remaining  two  GAO  recommendations  essentially  involve  mechanical 
kinds  of  processes.  One  is  to  establish  a monitoring  and  feedback 
system  for  tracking  utilization,  and  the  other  is  to  develop  a 
management  mechanism  for  resolution  of  issues  between  the  researcher 
and  the  user.  We  will  be  addressing  these  issues  within  the 
Department  of  Defense. 

One  other  effort  that  directly  impacts  the  job  you  have  to  do  here 
is  the  1976  Defense  Science  Board  Task  Force  on  Training  Technology. 
There  are  three  or  four  important  conclusions  from  their  report  that 
I would  like  to  submit  to  you. 

The  first  and  foremost  is  again  one  that  was  mentioned  hy  Admiral 
Metcalf--the  need  for  cost-benefit  and  performance-effectiveness 
analysis  relating  to  the  use  of  R6D  results.  Cost-effectiveness 
has  become  a "buzz  word,"  but  my  impression  in  reviewing  the  programs 
of  the  military  departments  is  that  it  is  being  taken  seriously.  It 
costs  money  to  do  a good  cost-benefit  analysis;  nevertheless,  you 
see  more  and  more  RSD  programs  including  it  at  the  outset  as  one  of 
their  tasks  toward  the  utilization  of  research.  I think  this  is 
clearly  a step  in  the  right  direction. 

As  Captain  Clarkin  indicated,  the  Laboratory  Utilization  Report  also 
identified  problems  of  interface  and  utilization.  They  ini-luded 
recommendations  like  collocation  and  better  communication,  which 
I've  already  touched  on.  The  Laboratory  Utilization  Report  did 
find  that  the  human  resources  laboratories  offered  a unique  technical 
input  into  military  planning  and  decision-making.  It  is  our  Job 
here  at  the  utilization  conference  to  see  what  steps  can  be  taken  to 
make  that  i iput  more  effective  and  more  widely  used. 

Utilization  pervades  all  aspects  of  RSD  management.  I think  it  is 
essential  to  have  a good  investment  strategy  that  indicates  you  are 
working  on  important  problems.  Our  technology-base  request  for 
human  resources  research  to  Congress  this  year  was  over  $100  million, 
up  from  $37  million  in  the  early  1970's.  That  is  tremendous  growth. 


At  all  levels,  both  in  the  management  and  utilization  of  R&D,  we 
must  make  sure  that  we  have  a proper  and  well-thought-out  investment 
strategy  for  the  utilization  of  these  resources,  so  that  we  get  a 
maximum  return  on  the  dollars  invested.  These  are  high  leverage 
dollars.  They  address  problems  that  are  critical  to  the  Department 
of  Defense,  problems  that  relate  not  only  to  the  high  cost  of  manpower 
training,  but  also  to  the  actual  combat  readiness  of  our  forces. 
Decisions  are  going  to  be  made  about  training,  personnel  and  manpower. 
If  we  don't  have  the  information  available  to  assist  in  making  those 
decisions,  they  will  be  made  without  the  input  of  the  R&D  community. 

I would  like  to  emphasize  the  importance  of  having  users  involved  in 
all  phases  of  R£D.  Users  should  assist  the  research  community  in 
developing  more  quantified  statements  of  the  problems  that  they  are 
attempting  to  attack  in  their  research  and  development  efforts. 

Clearly,  this  is  easier  to  do  in  connection  with  an  advanced 
technology  demonstration  project  than  it  is  for  a basic  research 
project,  but  I think  we  need  to  think  in  quantitative  terms,  to 
remove  as  much  of  the  vagueness  from  our  requirements  as  we  possibly 
can.  Users  need  to  be  effectively  involved  not  only  in  the  generation 
of  requirements  but  also  in  the  RDT6E  planning  process. 

The  transition  of  research  results  to  actual  implementation  again 
requires  long  lead-time  in  terms  of  planning,  budgeting  and  allocating 
the  particular  kinds  of  resources  that  are  needed.  Without  appropriate 
cost-benefit  analysis,  it  is  very  difficult  to  convince  a manager,  a 
user,  or  a consumer  to  implement  a system  that  is  going  to  cost  him 
several  million  dollars  when  he  has  one  that  is  working  right  now. 

If  we  are  going  to  change  and  Improve  the  system,  we  have  to  show 
the  consumer  how  it  will  benefit  him.  And  he  has  to  be  willing  to 
trade  off  near-term  investment  cost  for  long-term  savings.  This 
is  probably  one  of  the  toughest  problems  we  face  because  the  system 
is  built  to  work  against  doing  that.  Congressional  review  cycles 
and  investment  in  DoD  focus  on  the  near-term  investment  cost  and 
not  the  overall  life-cycle  cost.  I hope  you  will  be  able  to  come 
to  grips  with  this  problem  during  your  deliberations  here. 

Another  aspect  of  the  R&D  process  is  one  in  which  the  conflict  of 
interest  between  the  researcher  and  the  user  is  probably  greatest. 

That  is  effective  user  involvement  during  the  ongoing  research. 

The  researcher  doesn't  want  the  user  telling  him  what  to  do,  and 
the  user  doesn't  want  a product  that  is  of  no  use  to  him.  So 
there  has  to  be  a mutual  interaction.  I can't  define  in  any  precise 
terms  what  this  interaction  should  be,  but  i think  there  needs  to  be 
compromise  on  both  sides. 

During  your  deliberations,  I strongly  urge  and  encourage  you  to  look 
at  the  whole  R&D  process,  not  just  the  utilization  end.  If  you  start 
there,  you  are  dead  before  you  start.  You  need  to  start  at  the 
planning  stages  in  looking  at  quantitative  requirements.  You  need 
to  look  at  user  involvement  during  all  stages  of  the  research  effort. 
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In  perspective,  I believe  we  can  reasonably  say  that  DDR&E  supports 
the  need  for  R&D  in  the  human  resources  area,  as  well  as  the  need  for 
better  and  more  visible  utilization  of  the  research  results.  Without 
better  utilization  the  money  well  is  going  to  dry  up,  and  we  believe 
that  this  would  have  a significant  and  critical  negative  impact  on 
our  combat  readiness.  Once  again,  I applaud  this  utilization 
conference  and  the  issues  that  have  been  laid  out  before  you.  Let's 
hope  that  we  can  come  up  with  some  concrete  conclusions  and  recommen- 
dations. I wish  you  luck  as  you  proceed,  and  I look  forward  to 
both  a near-term  and  a longer-term  follow-up  of  this  very  ambitious 
effort . 

DR.  SANDS:  On  behalf  of  Captain  Clarkin  and  others,  I'd  like  to  thank 
Admiral  Metcalf  and  Colonel  Taylor  for  their  thorough  and  insightful 
discussion  of  problems  and  issues  that  are  currently  before  us  as 
we  look  at  research  utilization  in  a hard  and,  I hope,  meaningful 
way.  Now  we'd  like  to  open  the  session  for  discussion. 


DISCUSSION 

DR.  SINAIKO:  Do  you  have  any  examples  of  particularly  good  utili- 
zation? 

COL  TAYLOR:  I think  there  are  examples  around.  We  routinely  try 
to  look  at  payoff;  unfortuantely , many  times  we  look  at  payoff  in 
terms  of  the  development  effort  itself  rather  than  the  actual  imple- 
mentation of  the  work  in  operational  commands.  At  any  rate,  one 
good  example  of  utilization  is  the  Army  program,  REALTRAIN.  This  is 
a program  that  is  intended  to  put  realistic  training  into  field 
maneuvers  for  the  Army.  It  consists  of  a feedback  mechanism  that 
uses  very  simple  equipment  but  is  based  on  results  that  have  come 
out  of  the  research  community  over  the  years  in  terms  of  knowledge 
of  results,  feedback,  and  repetition  of  training.  Essentially, 
each  combat  soldier  is  assigned  a two-digit  number,  which  is 
displayed  on  his  helmet,  and  each  offensive  soldier  is  given  a scope 
on  his  rifle.  The  technique  involves  the  offensive  soldier 
identifying  the  number  and  calling  it  out  to  a controller  who  takes 
the  person  spotted  out  of  the  field  exercise.  Once  the  exercise 
is  completed,  there  is  a discussion  and  review  of  the  training 
effort,  and  the  exercise  is  then  repeated.  Thus,  you  have  feedback, 
discussion  of  proper  and  improper  tactics,  and  repetition  of  the 
drill.  The  Army  is  developing  sophisticated  hardware  in  the  form 
of  lasers  attached  to  the  guns  and  an  identification  system  that 
would  consist  essentially  of  retro-reflectors,  so  that  the  system 
will  become  more  automated.  This  is  currently  being  implemented. 
REALTRAIN  grew  out  of  the  exploratory  development  program  and  it  has 
continued  to  receive  R&D  support  during  the  implementation. 

There  are  several  essential  elements  in  REALTRAIN  that  make  good 
utilization  possible.  For  one,  it  deals  with  a very  clear-cut 


operational  prob1em--the  problem  of  how  to  provide  safe  and  realistic 
training  for  troops  in  the  field  in  order  to  reduce  initiai  conflict 
casualties  in  the  event  of  war.  The  Army  presently  has  either  live 
firing  at  stationary  targets  or  more  interactive  field  exercises 
that  involve  firing  blanks,  with  the  controller  deciding  who’s  dead 
and  making  judgments  about  combat  effectiveness.  No  real  assessment 
is  made  of  the  casualties  involved  in  the  conflict,  so  that  the 
learning  becomes  very  stilted.  It  is  questionable  whether  or  not 
there  is  any  effective  learning  in  this  at  all.  Motivation  of  the 
troops  is  very  low. 

So,  REALTRAIN  looks  at  a real  problem;  it  has  user  involvement  'n 
all  phases;  and  the  R£0  community  is  involved  in  test  and  evaluation 
and  in  the  continual  improvement  and  update  of  the  program. 

DR.  HAMERUS:  Involvement  of  users  in  research  calls  fc  the  ability 
to  identify  or  define  users  in  a way  that  is  decision-related.  What 
is  the  likelihood  of  an  honest  effort  to  address  the  utilization 
problem  through  reorganization  of  the  research  and  development 
activity  in  the  military  relative  to  the  linkages  within  the  user 
agencies?  That  is,  users  at  a command  level  need  a certain  kind 
of  involvement  in  the  R&D  process  but  at  lower  decision  points  users 
need  different  involvement.  What  is  the  likelihood  of  the  military 
being  able  to  consider  reorganization  of  the  research  and  development 
structure  to  provide  closer  affiliation  with  the  user  structure,  so 
that  decisions  in  the  R&D  process  are  more  closely  tied  to  the  user 
control  point  rather  than  the  research  control  point?  In  that  way, 
the  user  would  determine  when  the  research  commences  and  who  con- 
tributes to  the  planning  decision  process  and  the  implementation 
process.  Accountability  would  be  easier  to  pinpoint  through  that 
kind  of  structure  rather  than  there  being  two  or  more  independent 
agencies  that  negotiate  with  one  another  about  whether  they  get 
i nvol ved  or  not . 

ADM  METCALF:  Let  me  see  if  I can  get  at  what  you're  saying.  The  way 
it  came  across  to  me  is  that  you  are  asking  (in  part),  "What  is  the 
probability  of  getting  the  senior  user,  who  I represent,  involved 
in  the  R&D  process?"  I assure  you  that  if  senior  users  get  involved, 
you  will  get  junior  users  involved.  That  happens  to  be  the  way 
things  work  in  our  marvelous  oligarchy.  You've  hit  the  right 
point;  I think  that  guys  1 i ke  me  need  to  have  sonie  degree  of  involve- 
ment in  research. 

Then  I think  you  raised  the  interesting  problem  of  who  you  talk  to 
at  the  lower  level  decision  points  about  such  things  as  exploratory 
development.  That  is  a very  definite  and  very  difficult  problem. 

All  I am  suggesting,  s i mpl i st ical I y I guess,  is  that  we  devise  a 
procedure  so  that  when  we  are  designing  research  we  have  to  take 
that  question  into  account  and  seek  to  answer  it  explicitly.  In 
some  cases,  the  answer  to  that  question  may  be  unknown.  Right  now, 
particularly  when  we  are  talking  about  funding  a given  piece  of 
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research,  we  don't  know  how  it  is  going  to  be  implemented.  Thus, 
whether  we  realize  it  or  not,  we  answer  the  question  by  making  a 
nondecision,  but  we  don't  make  it  an  explicit  nondecision.  I 
think  we  need  to  be  more  careful  in  our  planning  and  make  it  more  of 
a structured  process,  so  that  we  are  explicit  about  such  things  as 
the  research  plan,  what  it  is  going  to  cost,  what  the  expected 
outcomes  are,  and  the  risk  of  failure.  It  appears  that  lots  of 
researchers  are  unwilling  to  expose  the  fact  that  there  is  a risk 
of  failure.  They  don't  want  to  tell  me  that  I've  only  got  a 25 
percent  chance  of  success,  because  it's  their  bread  and  butter  and 
they  feel  that  quite  often  the  consumer  will  say,  "Hell,  I'm  not 
going  to  take  that  25  percent  chance,"  and  so  they  are  unwilling  to 
articulate  that  risk.  But  I think  that  somewhere  along  the  line, 
we've  got  to  educate  our  consumers  to  the  fact  that  research  is  risk, 
and  that  by  their  getting  into  the  process,  they  can  reduce  the 
risk.  I don't  know  whether  that  answers  your  question  or  not. 

DR.  HAMERUS;  Well,  it  speaks  to  it.  Just  one  more  little  piece 
on  that.  From  your  point  of  view,  what  would  be  the  reasonable 
likelihood  that  you  or  others  at  your  level  of  authority  might  be 
assigned  a certain  degree  of  responsibility  of  this  kind,  that  is, 
to  officially  "connect"  with  this  kind  of  research  involvement? 

ADM  METCALF:  Yes,  I think  that  is  probably  the  answer.  That  is 
why  I said  that  I ought  to  be  required  to  sign  off  on  R&D  projects. 
But  if  you're  going  to  get  the  consumer  like  myself  to  sign  off, 
the  other  side  of  the  coin  is  that  we've  got  to  know  what  we  are 
signing  off  on.  And  that  information  too  often  is  not  there. 

What  I'm  suggesting  is  that  we  explore  a means  of  giving  me, 
the  consumer,  a better  understanding  of  what  is  expected  out  of 
the  research;  what  its  side  effects  may  be,  what  risks  might  be 
involved,  what  the  probability  is  that  it  can  be  utilized,  and 
what  dollars  must  be  allocated  in  order  to  undertake  implementation. 
There  is  research  in  the  Navy  right  now  on  the  problem  of  improving 
our  recruiting  system.  It's  taking  place  right  here  at  NPRDC, 
and  it  looks  very  exciting.  The  problem  is  that  it  is  going  to  take 
a lot  of  dollars  to  implement  if  it  comes  to  fruition,  and  at  the 
present  time  there  isn't  a nickel  in  the  program  for  implementation. 

QUESTION:  What  you  are  saying  is  that,  to  be  complete,  a research  pi, 
requires  an  implementation  plan. 

ADM  METCALF:  Exactly. 

DR.  UHLANER:  I am  Jay  Uhlaner  of  the  Army  Research  Institute. 

Admiral  Metcalf,  I think  you  touched  on  my  question  but  I'd 
appreciate  further  elaboration.  You  correctly  mentioned  that  one  of 
the  problems  in  utilization  is  the  conflict  of  time  in  the  mind  of 
the  researcher  and  consumer.  Related  to  this  is  the  problem  of 
changing  policy  positions  as  different  individuals  come  into  the 
policy-making  roles.  For  example,  under  one  set  of  policies  the 


utilization  of  women  in  large  numbers  might  possibly  be  an  objective; 
under  another  Admiral  or  General  it  might  be  reversed. 

ADM  METCALF:  That  is  an  absolute  fact,  not  a conjecture. 


DR.  UHLANER:  How  do  you  see  the  research  community  adjusting  itself 
to  those  major  policy  changes  if  the  consumer  is  going  to  determine 
the  needs  as  you  suggest?  Do  you  suggest  that  programs  be  reviewed 
as  new  policy-makers  come  in  and  be  cancelled  if  they  are  inconsistent 
with  the  new  policy-maker's  objectives?  I think  this  is  related  to 
your  comment,  if  I understood  you  correctly,  that  there  is  no  need 
for  6.1  and  6.2  efforts  in  this  area. 

ADM  METCALF:  No,  no,  no.  I was  misunderstood;  I think  there  is  a 
very  definite  need  for  6.1  and  6.2  programs.  What  I was  saying 
with  my  comment  on  6.1  and  6.2  (and  I was  really  referring  all  the 
way  through  6.5)  is  that  bureaucratically  it  is  designed  for  hardware 
research.  I find  it  difficult  to  structure  human  research  programs 
using  that  framework.  I think  you  absolutely  must  have  6.1  and  6.2. 

It  is  a most  difficult  area  to  define,  but  I think  that  if  you  can 
figure  out  some  way  to  design  an  implementation  plan  for  6.1  and  6.2 
that  may  answer  your  other  question.  A lot  of  research  with  regard 
to  utilization  of  women,  for  example,  is  policy  driven.  Thus,  some 
types  of  research  on  this  matter  may  not  be  addressing  the  underlying 
consideration.  That  is  a risk.  In  large  measure,  the  research 
community  is  a service  organization.  You're  like  the  supply  officer 
or  the  logistician  in  the  Army,  Air  Force  or  Navy  who  has  gone  out 
and  bought  all  kinds  of  propellers  for  his  airplanes;  and  the  next 
year  we  go  to  jets  and  we  don't  need  propellers  anymore.  You  are 
undertaking  research  into  certain  aspects  of  male  chauvinism,  which 
is  dissolving.  It  may  be  that  we  won't  need  those  solutions  anymore. 
This  is  a risk  in  project  selection. 

Now,  should  we  review  all  R&D  projects  every  time  we  get  a new  policy 
change?  You  know,  I think  in  some  ways,  that  is  the  role  of  the  R£D 
manager;  it  is  a very  important  thing  to  consider.  In  large  measure, 
the  R&D  manager  has  isolated  the  researcher  from  the  whims  and 
changes  of  the  policy-maker  and  properly  so.  (Sometimes,  however, 

I think  he  is  over i solated. ) But  that  is  one  of  the  manager's 
functions;  he  serves  as  a very  useful  bumper.  The  research  manager 
usually  has  a long  term  in  his  job.  He  is  used  to  policy  or  leader- 
ship changes.  Thus,  if  he  sees  that  a given  research  project  is 
likely  to  have  some  productive  outcome--if  it  is  solving  the  needs 
of  a plan  which  we  set  out  before  current  policy  changes,  but  which 
will  have  reasonable  potential  for  utilization,  he  should  say  whether 
it  should  continue  or  not.  I'm  not  sure  I could  turn  off  research 
that  is  going  on  right  now  even  if  I wanted  to  as  a consumer.  And 
I'm  not  so  sure  that  that  is  altogether  bad. 

DR.  UHLANER:  Right.  It  seems  to  me  that  one  part  in  the  total  trade- 
off in  this  equation  that  hasn't  been  mentioned  this  morning  is  some 
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concern  with  the  quality  of  the  researcher  that  you  can  attract  to 
the  program.  There  must  be  some  continuity  of  incentive  other  than 
just  concern  about  personal  survival.  I think  the  survival  of  RSD 
as  a creative  field  needs  to  be  based  more  on  the  brilliance,  the 
creativity,  the  capability  and  the  outstanding  work  of  the  research 
community  than  is  possible  if  it  is  driven  by  the  day-to-day  consumer 
requirements.  It  seems  to  me  that  we  have  to  find  a way  of  balancing 
off  one  with  the  other.  If  one,  such  as  the  immediate  demands  of 
the  consumer,  drives  out  or  significantly  reduces  the  likelihood  of 
attracting  excellent  scientists  who  might  be  interested  in  the  field, 
we  have  a counterproductive  system.  I'm  looking  at  it  from  a total 
systems  point  of  view.  I haven't  heard  any  concern  with  the  quality 
of  effort  and  how  it  might  relate  to  this  whole  interrelationship 
with  the  consumer. 

ADM  METCALF:  I would  submit  that  you  have  an  excellent  point.  It 
happens  to  be  a point  that  is  made  over  and  over  again,  namely, 
that  controls  on  research,  such  as  requirements  for  accountability 
and  relevance,  may  make  life  somewhat  nervous  for  researchers.  I 
do  not  feel  sympathetic  with  research  just  for  the  sake  of  research, 
which  is  what  I think  you're  talking  about.  In  other  words,  I'm  on 
the  other  end  of  the  spectrum  of  what  you  have  just  described.  Very 
often,  getting  highly  creative,  productive  people  is  a tough  problem. 
But  we  should  not  let  the  problem  of  getting  good  people  overshadow 
the  obligation  for  planning,  accountability  and  relevance. 

Thus,  what  I hear  you  saying  is  at  one  end  of  the  spectrum  and  I 
have  just  described  the  other.  Now,  somewhere  along  the  line,  we've 
got  to  have  a meeting  of  minds  on  this  legitimate  question.  Probably 
it  is  going  to  be  compromise. 

DR.  UHLANER:  I'd  like  to  return  to  the  example  that  Colonel  Taylor 
cited  as  an  excellent  illustration  of  R&D  utilization  that  came  out 
of  our  Institute,  namely  REALTRAIN.  The  emphasis  in  the  beginning 
of  that  program  was  technology  based,  not  at  all  specified  by  the 
user.  The  user  was  informed  informally  about  our  research  effort. 

In  fact,  the  general  officers  who  witnessed  it  at  one  of  the  so-called 
"shootoffs"  said,  "Gee  that's  enough  exploratory  development;  let's 
get  it  into  the  system  right  now."  So,  I think  that  although  my 
words  emphasized  the  other  thing,  it  may  be  pointing  toward  the 
very  objective  we  all  want  to  get  to,  namely,  developing  good 
products  for  the  user. 

ADM  METCALF:  Let's  look  back  for  a second.  You  thought  that  you 
had  a good  program  going  and  you  carried  it  out  from  exploratory 
development  funds.  Does  that  mean  that  at  the  start  of  such  a 
program  you  couldn't  sit  down  with  somebody  in  the  appropriate 
bureaucracy  and  get  their  sponsorship? 


DR.  UHLANER:  It  probably  would  have  been  rejected. 


ADM  METCALF:  Are  you  sayinq  that  if  you  invite  joint  planning  and 
review  of  an  exploratory  development  project  by  a potential  sponsor/ 
user,  the  risk  of  rejection  is  too  high? 
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DR.  UHLANER:  No,  I think  it  is  because  the  kinds  of  risks  that 
are  perceived  by  a potential  user  before  actual  demonstration  and 
tryout  seem  to  him  to  be  very  high  risks.  It  is  only  after  demon- 
stration that  the  user  can  get  a better  feel  for  whether  or  not  the 
risks  are  within  the  limits  of  what  he  can  accept.  So  I think  you 
have  to  deal  with,  and  I think  the  working  sessions  will  have  to 
deal  with,  the  very  important  concept  which  you  elucidated,  namely, 
risk  taking.  Can  you  think  of  some  examples  of  high-risk  projects 
that  appealed  to  you,  that  is,  where  high  risk  was  taken  and  you 
were  glad  it  was?  The  low-risk  projects  will  correlate  with  mediocre 
and  mundane  advancement,  by  and  large.  But  it  is  the  high-risk  ones 
that  tend  to  be  more  innovative,  and  are  likely  to  have  more  impact. 
It's  the  same  principle  as  the  stock  market.  If  you  are  going  to 
take  a long-shot  risk,  you're  likely  to  lose  a lot  more  or  gain  a 
lot  more.  It  is  the  safe  stock  that  will  give  you  maybe  a small 
appreciation  or  a small  loss.  You  have  the  same  thing  in  the 
research  business.  Can  you  think  of  some  high-risk  ones  where  you 
as  a consumer,  ahead  of  the  game,  said,  "Gee,  let's  go." 

ADM  METCALF:  I'm  working  of  a high-risk  problem  right  now;  I'm 
trying  to  reorganize  the  Bureau  of  Naval  Personnel.  (Captain  Clarkin 
thinks  he  did,  but  I've  got  news  for  him.)  The  risk  problem  is 
a very  difficult  one.  All  I'm  suggesting  is  that  if  we  base  every- 
thing on  the  assumption  that  the  user  is  going  to  reject  high  risk, 
we've  got  a real  problem.  But  if  you  recall,  I also  said  we've 
got  to  educate  the  user.  In  my  own  case,  if  I know  what  the  risk  is, 
or  at  least  have  some  quantification  of  it.  I'll  accept  that. 

Possibly  my  attitude  is  unusual,  since  I've  been  in  R&D  for  some 
time,  so  I'm  used  to  the  concept  of  risk  taking  when  investing  in 
research.  That  is  a problem,  but  it  doesn't  mean  that  we  can't 
discipline  ourselves.  Maybe  the  answer  is  to  make  the  plan  and  not 
show  it  to  anybody.  Then  when  a useful  product  or  procedure  turns 
up,  by  golly,  you've  got  a good  idea  here  and  we  users  say  to  the 
R6D  managers,  "Well,  how  are  we  going  to  implement  this  thing?"  The 
researcher  then  breaks  out  his  plan  and  says,  "Hey,  listen,  this 
is  how  we  thought  it  would  go  before  we  started."  You  know,  maybe 
it  only  goes  as  far  as  the  R&D  manager;  we  don't  know.  That  is  why 
I say  that  the  RSD  manager  has  a very  key  role  in  this  whole 
equat i on . 

DR.  UHLANER:  You  know,  I'm  glad  you  gave  that  answer,  suggesting  that 
we  need  a very  high  level  of  R&D  managers. 

CAPT  CLARKIN:  The  problem  you  cite.  Jay,  is  a very  real  one.  I 
think  it  goes  back  to  the  several  armed  camps  that  the  Admiral 
addressed  in  his  presentation  — the  camps  of  the  users,  researchers, 
and  the  research  managers.  I think  each  of  these  camps  views  the 
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others  pejoratively,  and  I think  one  effort  that  will  contribute  to  a 
solution  is  greatly  increased  interaction  among  all  of  those  camps. 

I think  the  operational  world  generally  views  the  researchers  and 
their  world  as  being  populated  by  fools  and  charlatans,  and  the 
researchers  generally  view  the  operational  world  as  being  seif- 
aggrandizers  who  temporarily  occupy  positions  where  they  can  get  a 
punch  check  on  their  card.  How  the  research  managers  view  both  of 
these  camps,  I'm  not  going  to  address  for  fear  of  being  sued.  The 
operational  world  is  concerned  with  fire-fighting,  damage  control 
issues,  today's  problems;  the  researchers  are  concerned  generally 
with  the  application  of  the  technologies  that  are  emerging  or  pre- 
sently existing.  I believe  that  with  increased  interaction,  the 
operational  world  can  responsively  exercise  the  degree  of  autonomy 
that  is  implicit  in  what  you've  stated. 

I further  believe  that  the  chief  executives  in  the  operational  world 
can  be  enlisted  to  support  the  R&D  community  as  a function  of  the 
degree  of  success  that  they  experience  from  that  process,  and  as  a 
function  of  that,  they  can  arrive  at  what  I've  characterized  as  a 
willing  suspension  of  disbelief  in  what  it  is  that  the  R&D  people 
were  doing.  I can  recall  specific  examples  where  that  willing 
suspension  of  disbelief,  or  perhaps  unawareness,  has  produced 
products  that  were  urgently  needed  at  the  time  and  that  were 
present  at  the  time  they  were  needed.  One  of  them  that  I've  cited 
before  originated  from  the  ONR  community  where  Likert's  work  was 
funded  in  the  latter  Ii0's,  eagerly  sought  after  by  industry  and 
widely  applied.  It  was  27  years  later  before  the  Navy  ever  became 
aware  of  what  the  product  consisted  of  and  then  aggressively 
implemented  it  in  its  human  resources  management  centers.  The  other 
work,  that  I think  many  in  this  room  participated  in,  is  the 
development  of  computer-assisted  instruction  in  the  50‘s.  Had  that 
been  submitted  for  control  by  the  operational  people  (who  at  that 
time  were  luxuriating  in  vast  amounts  of  money,  as  well  as  a never- 
ending  supply  of  people  to  populate  their  classrooms  and  to  present 
the  instructions),  they  would  have  rejected  it  out  of  hand.  But 
because  there  was  a degree  of  autonomy  in  the  R&D  community,  they 
were  able  to  pursue  that  innovation  and  have  it  available  at  the 
time  when  the  people  and  money  problems  were  much  more  dire  than 
they  were  at  the  time  the  development  was  initiated. 


INTRODUCTION  OF  BANQUET  SPEAKER 


Dv.  James  J,  Regan 
Technical  Director 
Navy  Personnel  R&D  Center 


Those  of  you  who  received  your  meals  later  than  the  rest  of  us, 
please  eat  faster  and  more  quietly.  We  have  a problem  when 


establishing  a schedule.  Dr.  Frank  Sands,  whom  I'm  sure  you'll  agree 
has  done  a heroic  job  in  arranging  for  this  symposium,  may  have  gone 
a bit  overboard  in  setting  a schedule  for  this  evening's  proceedings, 
but  we  have  one.  I'm  assuming  that  you've  forgotten  to  bring  this 
little  booklet  because  if  you  didn't  forget  to  bring  it,  there  is  almost 
no  reason  for  my  being  up  here. 

My  real  function  is  to  provide  a few  remarks  during  the  time  you  are 
finishing  your  roast  beef  or  whatever  you  got.  If  you  asked  for  fish 
and  you  didn't  have  the  grey  card  you  probably  didn't  get  it.  As  a 
matter  of  fact,  if  you  had  the  grey  card,  you  probably  didn't  get  it 
either.  But  next  year  we  are  going  to  give  you  some  options. 

We  are  on  something  of  a schedule,  and  I'll  try  to  maintain  it.  You 
may  not  be  familiar  with  the  schedule  because  it  is  written  in  a curious 
set  of  numerics  which  I'm  not  going  to  attempt  to  translate,  and  since 
you  don't  have  the  book  it  won't  make  any  difference  anyway.  I had 
been  asked  by  several  of  you,  however,  during  phase  1 of  this  three- 
phased  operation,  whether  we  were  going  to  have  an  invocation.  It 
probably  has  become  clear  to  you  by  this  time  that  we're  not  going 
to  have  one.  Dr.  Sands  did,  I must  confess,  want  an  invocation,  some- 
thing that  v/ould  make  our  "spirits  soar,"  I believe  were  his  terms. 

Since  we  had  quite  a few  spirits  during  phase  1,  I thought  we  were 
all  soaring  already,  so  I wasn't  too  concerned  about  it.  But  he  persisted 
and  suggested  that  perhaps  a passage  from  the  gospel  according  to  the 
Harvard  Business  School  by  Jim  Clarkin  would  be  in  order.  But  since 
it's  so  sectarian  and  there  are  so  many  nonbelievers,  I didn't  think 
that  would  be  appropriate  either.  I have  a number  of  eminently  forgetable 
invocations,  which  incidentally  I've  forgotten,  so  I'm  not  going  to 
give  you  one.  Therefore,  we're  not  going  to  have  an  invocation. 

What  we  are  going  to  have,  however,  very  fortunately,  are  some  remarks 
by  Dr.  Rostker.  And  I guess  among  other  things  I'm  not  going  to  do 
this  evening  is  read  a list  of  his  accomplishments  and  background, 
for  several  reasons,  one  of  which  is  that  you  have  them  in  your  program, 
which  you  probably  don't  have  with  you  either.  But  in  the  morning, 
if  you  can  remember,  you  can  look  at  the  program  and  you  can  learn 
about  his  background.  I should  say  that  he  is  an  economist  and  that 
he's  the  Principal  Deputy  Assistant  Secretary  of  the  Navy  for  Manpower, 
which  is  a title  almost  as  long  as  the  title  of  the  symposium.  So 
when  we  put  them  all  together,  we  couldn't  have  a little  program  as 
it  turned  out. 

I think  it's  important  to  say  again  (because  we  have  so  many  intra-meet- 
ings in  this  meeting,  nested  inside  nests,  we  might  have  forgotten 
why  we're  here  in  the  first  place)  that  our  purpose  is  not  to  reiterate 
for  the  nth  time  why  utilization  is  a problem  and  why  those  things 
that  are  utilized  are  not  credited  to  us  and  those  things  that  are  not 
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utilized  are  blamed  on  us--but  rather  to  arrive  at  some  sort  of  set 
of  specific  recommendations  of  a policy  nature  which  could  change 
this  world  about  which  we've  talked  so  much  for  so  long. 

I think  a step  in  that  direction  will  be  some  comments  we're  going  to 
hear  in  a few  moments  from  Dr.  Rostker.  Dr.  Rosker  is  a prime  member 
of  the  new  Administration  team,  who  I think  are  activists  and  very  techni- 
cally qualified  young  and  innovative  people  that  we'll  be  hearing  a 
great  deal  more  from.  Ordinarily,  I believe  people  say  that  our  featured 
speaker  needs  no  introduction  and  then  proceed  to  give  a long  one.  I'm 
willing  to  admit  that,  at  least  at  this  moment,  but  not  in  the  future, 
he  does  need  an  introduction  and  I'm  not  going  to  give  him  one.  I'm 
going  to  ask  him  to  come  up  here  and  say  what  he's  going  to  say. 

BANQUET  ADDRESS 


Dr.  Bernard  D.  Rostker 
Prinoipal  Deputy 

Office  of  the  Assistant  Secretary  of  the  Navy 
for  Manpower  and  Reserve  Affairs 


That's  a very  hard  act  to  follow,  I'm  afraid.  Originally  these  re- 
marks of  mine  were  to  be  the  keynote  speech,  but  we  had  a little 
problem  in  Washington  and  I didn't  quite  get  off  on  time,  and  I thank 
Joe  Metcalf  for  sitting  in  this  morning--we  sort  of  played  switchies. 

In  fact,  I actually  have  some  prepared  remarks  which  probably  were 
more  appropriate  as  a keynote  address  than  an  after  dinner  address,  but 
so  be  it. 

Let  me  give  you  my  perception  about  people-related  P&D  programs  and 
their  management.  In  my  opinion,  research  is  only  as  effective  as  its 
contribution  to  the  solution  of  problems.  I believe  it  can  be  summed 
up  by  saying  that  research  should  be  relevant  and  it  should  be 
problem  oriented.  My  assessment  of  the  personnel  R6D  business  is 
that  a great  deal  of  the  work  done  today  is  neither  relevant  nor 
very  useful.  I see  few  lines  of  communication  between  the  researchers 
and  the  decision-makers.  I see  little  communication  in  either  direction 
between  the  researcher  and  ultimate  customer.  I do  not  see  enough 
follow-through  from  research  to  application  and,  like  it  or  not,  I 
think  the  person  most  responsible  for  this  situation  is  the  researcher 
and  the  research  community.  What  I'd  like  to  do  this  evening  is 

explore  with  you  some  of  my  thoughts  on  the  problem  and  suggest  a i 

few  things  that  I think  we  all  can  do  to  correct  the  situation,  and 
tell  you  some  of  the  things  I intend  to  do  from  vhere  I sit. 
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It's  often  said  that  the  [lepartment  of  Defense  spends  about  12  billion 
dollars  on  R6D  efforts.  We  also  tend  to  note  that  something  like 
one-tenth  of  one  percent  is  spent  in  personnel -related  research.  And 
it's  often  argued  that  this  is  evidence  for  the  fact  that  we  should 
increase  the  size  of  the  personnel  R6D  programs.  However,  at  this 
point  in  time  I'm  more  concerned  with  what  we  are  getting  for  the 
money  we  are  spending  rather  than  arguing  for  an  increase  in  the 
size  of  the  program.  And  I would  submit  to  you  that  we  are  getting 
less  than  fifty  cents  on  the  dollar,  maybe  less  than  the  twelve 
cents  on  the  dollar  that  we  talked  about  earlier.  What  really 
concerns  me  here  is  not  that  we  produce  research  that  is  relevant 
and  that  for  reasons  beyond  our  control  it's  not  applied.  What 
concerns  me  is  that  so  much  of  the  research  going  on  is  irrelevant 
and  even  before  the  research  is  carried  through,  it's  a fait  accompli 
that  it  could  never  be  applied  and  it's  really  not  directed  toward 
any  practical  problems.  I'm  not  at  all  suggesting  that  we  want  to 
have  a dollar  return  for  a dollar  spent  in  the  R6D  area.  A certain 
amount  of  theoretical,  uninhibited,  and  independent  research  is 
necessary;  but  I really  hope  I don't  have  to  remind  you  of  the 
intense  competition  that  exists  today  for  the  defense  dollar.  I 
would  submit  that  the  time  is  upon  us  to  face  this  issue  squarely. 

The  key  word,  I think,  is  relevance.  Your  efforts  must  be  structured 
much  more  to  the  solution  of  major  policy  questions  and  operational 
problems  than  to  the  requirements  of  your  discipline  and  the  acceptance 
of  your  peers.  We  must  be  responsive  to  the  needs  of  customers,  we 
must  learn  how  to  communicate  our  results  to  people  who  can  carry  them 
through  and  make  decisions;  in  this  respect  we  are  currently  not 
doing  a very  good  job.  The  most  important  contributions  that  we 
can  make  to  achieve  utilization  is  to  focus  our  work  on  the 
problems  at  hand,  to  carry  out  the  work  in  a comprehensive  manner 
which  draws  together  the  many  disciplines  that  are  represented  here. 

In  direct  terms,  utilization  will  result  only  if  we  work  the  problem, 
if  our  work  is  not  over  1 y theoret i ca 1 , and  if  our  approach  is  not 
fragmented.  While  we  certainly  need  proper  methodologies,  refined 
models,  and  sound  theory,  they  are  not  substitutes  to  a dedication 
to  working  the  Defense  Department's  problems  today.  The  factors  that 
contribute  or  constrain  the  cost  effectiveness  of  our  personnel 
research  efforts  are  widespread.  It  is  not  all  our  fault  that  our 
research  is  not  carried  through,  yet  the  R&D  community  can  do  much 
to  improve  the  present  situation.  Let  me  be  more  explicit.  I 
believe  we  are  constrained  today  by  the  very  nature  of  the  R£D 
structure  that  we  are  working  under.  The  definitions  of  6.1  through 
6.5  money,  the  hardware  orientation  of  the  terminology,  the  operational 
guidelines  and  the  practical  application  of  those  guidelines  are  not 
particularly  conducive  to  the  practical  application  of  R£D  in  a problem 
solving  mode. 

Now  let  me  digress  from  my  prepared  remarks  and  make  a side  comment. 
I've  spent  the  last  eight  years  at  the  Rand  Corporation  doing  R£D  for 
the  Air  Force.  But  we've  had  a unique  environment.  We  have  not 
been  subject  to  the  6.1,  6.2,  6.3  type  categorization;  we've  worked 


directly  for  the  decision-maker.  We've  been  able  to  take  problems 
and  carry  them  through  from  their  basic  definition  to  the  development 
of  whatever  unique  tools  are  necessary  to  get  the  required  answers 
through  the  application.  Coming  into  this  business  from  this  end, 

I've  found  it  really  disturbing.  The  types  of  discussions  that  we 
heard  about  today,  "is  this  6.1  money  or  is  this  6.2  money?"  "I 
can't  get  6.3  money,"  and  "should  the  research  be  required  to  go 
from  6.1  to  6.2?"  I have  a lot  of  trouble  with  that.  I believe  a 
structure  should  facilitate  solving  a problem,  and  I would  submit 
to  you  that  the  present  structure  that  we  have  in  the  R&D  community 
may  be  very  appropriate  for  the  hardware  world  but  is  not  facilitating 
the  development  and  the  carrying  out  of  articulated  R6D  programs. 

I would  also  suggest  that  the  present  structure  gives  the  researcher 
a convenient  place  to  hide  while  he  does  his  own  thing.  From  the 
researchers'  point  of  view  this  is  really  understandable.  I 
know  many  have  grown  accustomed  to  conducting  their  research  in  their 
own  environment,  in  their  own  areas  of  expertise,  free  of  outside 
interference  and  time  pressures.  Unfortunately,  researchers  have 
little  incentive  to  solve  practical  problems;  furthermore,  results 
are  delivered  to  the  services  usually  in  piecemeal  fashion  in  the 
apparent  hope  that  decision  makers  can  determine  how  best  to  integrate 
the  results.  Then,  as  if  to  ensure  nonimplementation,  researchers 
often  put  their  results  in  technical  reports  or  professional  journals. 
My  message  this  evening  is  that  this  is  a way  of  operation  that  we 
can  no  longer  afford. 

A contributing  factor  to  the  present  situation  that  we  find  in  the 
R&D  community  is  the  traditional  way  that  people  plan,  program  and 
budget.  My  main  concern  here  is  that  it  is  the  researchers  who  are 
involved  in  the  planning,  programming  and  budgeting  of  RSD  funds 
and  these  are  in  fact  the  wrong  people.  Working  up  my  remarks 
tonight,  I became  increasingly  concerned  about  the  domination  of 
researchers  in  the  R&D  planning  process.  As  far  as  the  Navy  is 
concerned,  I believe  that  the  people  who  should  be  most  involved 
in  the  development  of  the  research  program  are  the  customers  of  the 
research.  I come  to  this  conclusion  knowing  full  well  that  these 
people  have  often  not  provided  the  guidance  or  support  that  is 
needed  to  the  R&D  community,  but  this  is  a risk  that  I submit  is 
worth  taking.  If  we  cannot  develop  a research  program,  if  we  cannot 
have  the  research  program  supported  by  the  ultimate  consumers  of 
those  programs,  then  I would  submit  we  have  a difficult  time  making 
that  research  relevant  and  then  carrying  the  research  through  to  its 
ultimate  conclusion  and  application. 

For  those  of  you  who  have  not  seen  it,  I would  Invite  your  attention 
to  the  so  called  Ginzberg  Report  of  1971.  The  report  found,  among 
other  things:  (1)  a lack  of  communication  between  policy  and  decision- 
makers and  researchers,  (2)  too  much  control  of  research  by  the 
hardware  community,  (3)  too  much  research  coming  from  the  bottom 
up,  rather  than  the  top  down,  (A)  too  little  involvement  and  control 
by  those  who  are  responsible  for  the  program,  but  not  necessarily 
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responsive  to  the  needs  of  the  client,  and  (5)  too  much  concentration 
on  research  in  the  hands  of  particular  groups,  and  the  Ginnesburg 
report  singled  out  psychologists,  and  too  little  use  of  the  wide 
variety  of  disciplines. 

In  general,  I agree  with  these  findings.  I have  found  that  little 
has  changed  in  the  six  years  since  this  report  was  issued.  While 
the  need  for  manpower  research  has  never  been  more  pressing,  the 
support  that  human  resources  researchers  received  in  the  Congress 
has  never  been  less.  On  April  7th  of  this  year,  the  House  Armed 
Services  Committee  devoted  about  five  pages  to  the  need  for  personnel 
research.  The  House  Committee  highlighted  the  following  areas: 
they  saw  a need  for  a better  technique  in  forecasting  force  re- 
quirements; they  argued  for  more  research  to  come  to  grips  with  the 
declining  manpower  pool;  they  argued  for  more  research  to  develop 
optimal  physical  and  mental  standards;  research  on  the  increased 
utilization  of  women;  and  research  on  the  substitution  of  civilians 
for  military.  It  is  interesting  to  note  that  in  the  same  report 
the  House  also  slashed  many  of  our  RSD  programs.  In  the  Navy,  the 
Training  and  Human  Engineering  Technology  program  was  reduced  by 
seven  million  dollars  (from  nine  to  two  million  dollars).  The 
Human  Factors  Engineering  Program  was  reduced  from  three  million 
dollars  plus  to  zero,  and  the  Human  Effectiveness  Program  was  reduced 
from  about  five  million  dollars  to  about  two  million  dollars.  The 
message  that  the  House  was  presenting  was  clear.  While  many  of  the 
research  efforts  were  in  areas  of  interest  to  the  committee,  they 
felt  that  the  present  R&D  community  was  not  doing  the  job  and  was 
not  tackling  the  problems. 

Well,  where  do  I stand  on  these  issues?  I intend  to  work  toward 
putting  our  research  program  and  funding  in  the  hands  of  those  who 
set  the  lines  of  communication  between  the  decision-makers  and  the 
researchers.  I intend  to  see  that  at  least  in  the  Navy,  we  reorient 
our  RSD  efforts  more  toward  solving  problems.  My  message  is  really 
clear,  I'd  like  you  to  Join  in  these  efforts.  I doubt  that  ail  of 
you  or  even  many  of  you  here  will  be  enthusiastic  about  these, 
recommendations,  but  I would  submit  to  you  ladies  and  gentlemen,  that 
the  future  of  personnel  research  in  the  Defense  Department  is  at 
stake.  I hope  you  will  ponder  these  thoughts  carefully  in  the 
course  of  this  symposium.  If  you  do  not,  I strongly  believe  that 
personnel  R&D  will  cease  to  be  a viable  and  Justifiable  part  of  the 
Defense  budget.  If  this  occurs,  the  services,  and  certainly  DoD, 
will  be  the  losers.  But  I think,  more  importantly  or  equally 
importantly,  you  all  will  be  the  losers. 

I thank  you  very  much  for  the  opportunity  to  speak  here  tonight 
and  I'd  be  happy  to  answer  any  questions. 
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DISCUSSION 


CDR  DAVIS:  You  referenced  the  Ginzberg  Report  of  1971  and  you 
made  a comment  that  there  was  too  much  research  coming  from  the 
bottom  up  instead  of  from  the  top  down.  And  yet  I interpreted  the 
thrust  of  your  remarks  as  getting  the  user  involved.  Maybe  I 
don't  understand  your  perception  of  user,  as  opposed  to  mine,  which 
has  the  user  some  place  at  the  top. 

DR.  ROSTKER:  Well,  1 think  there  are  two  parts  to  that.  One  is 
that  there's  still  a great  deal  of  research  that  is  being  done  for 
research's  sake,  that  is  generated  by  the  research  community  and  is 
perpetuated  by  the  research  community.  We  get  into  very  comfortable 
modes,  and  I think  it's  only  natural  that  researchers  on  your  staff 
continue  to  do  the  things  they  know  best,  which  is  the  thing  they 
did  last  year,  but  that's  unacceptable.  That's  one  point  in  terms 
of  the  bottom  up. 

The  second  point  is,  I don't  accept  the  premise  that  the  customer 
is  at  the  bottom  of  the  heap.  I think  that's  one  of  the  real  problems. 
There  is  a tremendous  amounc  of  research  that's  necessary  in  a policy 
sense  that  can  really  have  an  Impact  on  the  Defense  Department,  that 
can  have  an  impact  on  the  Navy.  And  the  customer  for  that  research 
is  not  a naval  lieutenant  someplace.  The  customer  can  well  be 
an  admiral  who  is  trying  to  come  to  grips  with  pretty  large  policy 
questions  in  the  a.reas  I'm  concerned  about--manpower , personnel  and 
training.  I just  don't  see  why  the  consumer  has  to  be  a lower- 
level  person  within  an  organization.  1 got  a sense  of  that  as  the 
Air  Force  representative  was  talking  today  and  said  that  the  customer 
might  be  a captain  in  the  Air  Training  Command.  I spent  most  of  my 
professional  life  in  Air  Force  manpower,  and  I understand  that 
remark.  I understand  those  requirements,  and  I understand  the  kind 
of  research  that  is  done  in  response  to  that.  I would  also  submit 
that  the  Air  Training  Command  has  some  very,  very  major  management 
problems,  that  the  Air  Force  has  some  management  problems  in  training 
strategies,  and  that  those  problems  are  basically  not  being  looked 
at.  I would  be  concerned  that  the  kinds  of  skills  that  are  repre- 
sented in  some  of  the  R&D  organizations  are  not  very  conducive 
to  looking  at  some  of  the  broader  questions  of  systems  management, 
efficiency,  tradeoff.  There  are  plenty  of  customers  who  are  at 
high  policy  levels  who  need  help  badly.  And  I don't  think  we 
are  giving  it  to  them. 

DR.  UHLANER:  I think  it  would  be  very  helpful  to  me  if  you  could 
think  of  some  examples  anywhere,  whether  in  industry  or  in  other 
government  agencies,  where  a research  program  is  entirely  dictated 
by  the  consumer.  I'm  not  challenging  it--l'm  just  trying  to  think  it 
through.  You  need  a model,  but  as  I wrack  my  brain  I'm  having 
difficulty  thinking  of  one  area  where  the  consumer  alone  lays  out 
the  research  program.  Take  the  patient  in  the  hospi tal--does  he  lay 
out  the  research  program?  Or  take  any  industrial  sett i ng--does  the 
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driver  dictate  what  kind  of  automobiles  will  be  built?  I'm  not 
saying  it  shouldn't  be  tried,  but  I'm  trying  to  understand  it. 


DR.  ROSTKER:  Let  me  answer  your  question  from  my  own  background, 
and  of  course,  that's  why  I make  these  remarks.  The  organization 
that  directed  the  research  we  did  at  RAND  was  an  Air  Force  Advisory 
group  chaired  by  the  Deputy  Cnief  of  Staff  (DCS)  for  R6D  and  made  up  of 
the  Assistant  Deputy  Chiefs  of  Staff  of  each  of  the  functional 
organizations  on  the  Air  Staff.  That  research  program,  as  good  or 
as  bad  as  it  was,  was  done  in  response  to  the  research  needs  of  the 
DCS's  on  the  Air  Staff.  I ran  a program  that  was  responsive  to  the 
DCS  Personnel.  A certain  amount  of  the  program,  by  agreement,  was 
kind  of  free  in  what  I wanted  to  do,  but  most  of  the  program  was 
in  support  of  real  hard  requirements  and  problems.  Those  problems 
were  not  short  term.  We  consistently  avoided,  at  some  political 
expense,  the  six-month  turnaround  and  even  the  year  turnaround. 

And  we  were  able  to  argue  effectively,  when  necessary,  that  there 
were  needs  for  new  methodologies,  new  material,  new  theories,  and 
new  ways  of  applying  existing  techniques  that  were  groundbreak i ng . 

But  we  did  it  in  the  context  of  an  operational  problem  for  which 
there  was  an  operational  solution  down  the  road.  And  I think  it 
was  very  effective.  I think  it  was  effective  in  the  logistics  area 
and  in  the  strategic  area.  We  did  basic  research  in  areas  that 
might  be  called  operations  research.  We  did  some  groundbreaking 
in  statistics,  but  we  did  it  in  connection  with  an  operational 
problem  because  there  was  a real  need,  not  because  we  hired  a theoreti- 
cal statistician  to  go  out  and  do  his  thing  and  fund  him  on  that 
basis.  And  that's  what  I'm  really  shooting  at.  That's  frankly  what 
my  mode  1 is. 

DR.  UHLANER:  That,  then,  was  a joint  planning  effort. 

DR.  ROSTKER;  Yes,  it  was  a Joint  effort. 

DR.  UHLANER:  That  I can  understand. 

DR.  ROSTKER:  The  guidance  we  worked  under  was  that  the  program  would 
be  jointly  determined  by  RAND  management  and  the  Air  Force  Advisory 
group. 

COMMENT  FROM  THE  AUDIENCE;  I'm  having  some  difficulty.  I agree 
with  you  completely  about  the  need  for  the  researcher  to  get  into 
the  operational  problem  area.  The  thing  that  is  perplexing  me  is 
why  the  emphasis  is  on  high-level  policy  making  and  decision  making. 

You  apparently  are  suggesting  that  we  should  be  focusing  our  attention 
on  what  Congress  says  on  the  basis  of  some  evaluation  by  the  House 
Appropriations  Committee  of  the  total  spectrum  of  research  that's 
going  on  under  manpower  RSD,  which  I cannot  believe  is  a well- 
informed  set  of  considerations.  So  I'm  asking  whether  you  are 
really  saying  that  there  is  a set  of  priorities  we  should  be  responding 
to  that  are  at  the  very  high-level  policy-making  decision  levels.  If 
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help  was  needed  in  those  areas,  I'm  sure  that  almost  everybody  in 
this  room  would  be  delighted  to  plunge  in,  to  the  extent  that  we  felt 
it  was  possible  to  do  something  worthwhile.  On  the  other  hand, 
there's  a tremendous  amount  of  research  going  on  at  the  lieutenant 
and  captain  levels,  with  really  significant  day-to-day  operational 
problems,  which  is  going  to  make  a real  difference,  in  my  estimation, 
at  to  how  ready  the  Department  of  Defense  people  are  to  perform  their 
mission. 

DR.  ROSTKER:  I understand.  Let  me  answer  at  two  levels.  First,  I 
think  if  you  deal  in  terms  of  the  budget,  in  terms  of  the  critical 
decisions  that  are  going  to  make  a difference  to  the  defense  program 
over  the  next  ten  or  fifteen  years,  there  are  researchable  problems 
at  the  high  level  for  which  we  do  not  have  answers  and  which  des- 
perately need  good  research.  Congress  is  not  wrong  in  their  per- 
ceptions, and  the  people  who  are  in  the  decision-making  positions  in 
the  Defense  Department  and  the  Services--on  your  own  central  staffs, 
on  the  OPNAV  Staff,  the  Air  Staff,  the  Army  Staff--are  wrestling 
with  exactly  these  problems.  I'm  sure  there  are  good  research 
questions  at  the  captain  level.  The  question  is  whether  or  not  in 
a tight  budget  situation  we  can  afford  to  answer  the  captain's 
problems,  whether  we  can  afford  to  make  the  changes  at  the  bottom 
when  in  fact  we  have  tremendous  budgetary  pressures  at  the  top  and 
we  have  a need  for  research  at  the  top. 

There  is  also  a perception  problem.  With  the  unmet  needs  of  the 
Congress,  with  the  unmet  needs  of  the  top  decision-makers,  when  they 
start  looking  around  for  candidates  for  budget  cuts,  the  captain's 
not  standing  there  saying,  "That's  really  good."  And  frankly,  the 
press  that  the  human  resources  RSD  community  gets  from  the  average 
Navy  officer.  Army  officer.  Air  Force  officer  is  not  good  either. 

The  reputation  that  the  community  has  is  a reputation  of  doing  their 
own  thing,  being  impervious  to  the  needs  of  the  Defense  Department-- 
I ‘m  sorry  if  that's  the  case,  but  group  after  group,  study  after 
study.  Congressional  action  after  Congressional  action  is  pointing 
in  that  direction.  And  solving  Captain  Smith's  problems  at  a 
training  center  is  not  going  to  bale  out  this  community  in  terms  of 
increasing  problems  with  the  budget.  The  Navy  took  a six  billion 
dollar  cut  in  the  POM  program,  and  that's  going  to  get  translated 
right  up  and  down  the  line.  And  frankly,  the  person  that's  going 
to  make  those  determinations  is  going  to  make  them  based  on,  "What 
have  you  done  for  me  lately?"  And  if  you're  working  Captain  Jones' 
problems  down  at  the  training  center,  the  answer,  to  the  decision 
maker,  is  "Not  very  much." 

DR.  HAVELOCK:  I wonder  why  you  accept  at  face  value  what  the  Congress 
says  about  all  these  programs.  I've  been  trying  to  listen  around 
and  see  what  people  are  doing  and  what  kind  of  research  they  are 
supporting,  and  I don't  hear  them  supporting  the  kind  of  research 
you're  talking  about.  I wonder  why  you  take  the  view  as  essentially 
an  activist  for  the  Congress  on  this  issue. 
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DR.  ROSTKER:  Because  I happen  to  agree  with  the  Congress,  for  one. 

DR.  HAVELOCK:  What's  your  data  base  for  agreeing  with  the  Congress 
on  this  issue?  Give  us  a few  examples  of  the  research  that  is 
being  supported. 

DR.  ROSTKER;  It's  a matter  of  priority.  You  started  asking,  "Who's 
supporting  other  research?"  I would  submit  that  there  is  other 
research  being  supported.  For  instance,  last  year  the  Office  of  the 
Assistant  Secretary  of  Defense  for  Manpower  and  Reserve  Affairs  went 
to  the  Congress  to  create  a two  million  dollar  Manpower  Institute 
because  they  were  not  getting  the  kinds  of  research  to  these  kinds 
of  questions  that  they  felt  were  necessary.  Incidentally,  they  went 
in  and  asked  for  O&M  funds. 

DR.  HAVELOCK:  Is  that  wrong,  or  what? 

DR.  ROSTKER:  No,  I don't  think  that's  wrong.  I think  it's  indicative 
of  the  fact  that  these  are  researchable  questions  and  that  they  were 
not  able  to  get  the  kinds  of  answers  they  needed  from  the  existing 
resea  rch  commun i ty . 

COMMENT  FROM  THE  AUDIENCE;  I think  part  of  the  frustration  here  is 
the  issue  of  who  is  Congress,  and  that's  one  we  encounter  continually 
in  all  our  public  policy  efforts.  On  the  one  hand,  we  deal  with  that 
part  of  Congress  which  is  concerned  with  defense,  and  they  are  saying, 
"We  want  to  do  something  about  women  and  we  want  to  do  something  about 
attitudinal  development."  But  they  won't  support  research  on  those 
problems.  On  the  other  hand,  there's  the  part  of  Congress  that's 
concerned  with  HEW,  the  U.  S.  Office  of  Education,  the  National 
Science  Foundation,  etc.,  which  represents  a different  point  of 
view.  And  I think  one  of  the  problems  everybody  is  having  is  which 
aisle  of  Congress,  at  what  time,  are  we  listening  to  in  terms  of 
developing  public  policy.  And  it  seems  to  me,  Mr.  Secretary,  that 
as  a new  administration  coming  in  and  as  a new  effort,  part  of  your 
responsibility  is  going  to  be  to  help  the  Carter  administration 
articulate  where  we  are  in  terms  of  human  resource  development  and 
where  we  want  to  go.  We  can  probably  live  with  whatever  priorities 
are  assigned.  But  it's  the  Catch  22  of  deciding  who  Congress  is, 
and  where  their  public  policy  is  more  appropriate  than  the  public 
policy  of  the  executive  side  of  government,  that  causes  great 
frustration.  And  I think  that  your  efforts  to  articulate  that  are 
going  to  be  important. 

COMMENT  FROM  THE  AUDIENCE:  As  far  as  we're  concerned,  Congress  works 
by  committees,  so  Congress  is  the  committee  with  which  we  have  to 
deal.  More  specifically,  it  is  the  subcommittees  of  those  committees 
with  which  we  have  to  deal,  and  then  a little  bit  more  specifically, 
it  is  the  influential  members  of  those  subcommittees  who  are  viewing 
us  in  whatever  way  they  are  viewing  us  and  the  influential  staff 
members  working  for  those  influential  members.  That's  the  Congress  as 
far  as  we're  concerned.  I don't  know  if  you  agree  with  me. 
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DR.  ROSTKER:  Let  me  kind  of  reiterate  where  I'm  coming  from.  I'm 


saying  that  I think  that  the  R6D  product  of  this  community  should  be 
relevant  and  problem-oriented.  To  the  extent  that  it  is--and  can 
be  packaged  in  a way  that  makes  it  meaningful  to  a decision-maker, 
makes  it  meaningful  to  the  people  who  have  to  carry  out  policy  and 
wrestle  with  difficult  problems--!,  and  I think  others  in  the  new 
administration,  am  very  willing  to  support  the  R&D  community.  To 
the  extent  that  we  are  doing  pure  basic  research,  and  to  the  extent 
that  what  should  be  applied  research  is  not  being  applied  (because 
the  researchers  are  not  carrying  through  since  follow  through  is 
not  to  their  liking),  I wi 1 I not  support  it.  And  I'll  work  actively 
against  it. 

LCDR  MAIRS:  1 think  we've  had  a very  distinct  message  today,  first 
from  Captain  Clarkin,  followed  by  Admiral  Metcalf,  and  now  from  you. 
The  message  is  clear  that  the  RSD  community  is  not  responding  and  that 
further  attempts  to  elucidate  reasons--by  blaming  the  Congress,  or 
blaming  some  sort  of  national  spectrum  that  does  not  appreciate  good 
research--is  not  really  going  to  help  us  get  on  with  the  task  we 
started  out  with  today.  1 think  we  ought  to  stop  that  and  start 
developing  the  answers  to  the  three  problems  that  Captain  Clarkin, 
Admiral  Metcalf,  and  you  have  identified. 

CAPT  CLARKIN:  May  I append  a comment  to  Lee  Mairs'  statement.  And 
I think  A1  Himes  will  recall  Admiral  Watkin's  presentation.  On  each 
of  the  issues  you  mentioned.  Admiral  Watkins  cited  Navy  research  as 
making  the  contribution  that  prepared  the  Navy  to  deal  with  Congress 
and  to  deal  with  the  issues  that  were  most  pressing.  That  was  the 
source  of  the  contribution  that  resolved  the  problem  or  at  least 
ameliorated  the  problem  with  Congress. 

DR.  ROSTKER:  Yes,  sir. 

COMMENT  FROM  THE  AUDIENCE:  With  the  exception  of  the  6.1  efforts,  I 
am  not  aware  of  a single  program  that  is  not  related  to  a Navy 
problem,  and  so  1 find  myself  uncomfortable  with  hearing  you  talk 
about  research  that  we're  conducting  that  is  not  related  to  Navy 
operational  problems. 

DR.  ROSTKER:  Part  of  i t i s a matter  of  the  way  it  is  presented  and 
the  way  it  is  carried  through.  I had  the  pleasure  of  having  a number 
of  briefings  from  Jim  Clarkin  earlier  last  month.  Most  of  them  were 
first  rate,  but  I remember  one  in  which  a researcher  spent  about  15 
minutes  talking  about  a theoretical  model  of  work  and  occupations 
and  everything,  and  it  was  absolutely  nonoperat i ona 1 until  he  got  to 
the  hard  things  of  what  he  had  actually  done.  And  that  was  exciting 
to  me,  as  a person  wrestling  with  a problem.  Nov  I would  submit 
that  if  that  type  of  research  could  have  been  packaged,  presented 
and  made  relevant,  and  if  the  researcher  had  been  able  to  talk  in 
my  terms,  then  there  wouldn't  have  been  a doubt  in  my  mind.  A lot 
of  what  you  think  is  irrelevant  is  in  fact  relevant.  The  question 
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is,  how  is  it  sold,  how  is  it  packaged,  how  is  it  presented,  and 
what  is  the  follow  through?  And  if  we  don't  do  those  things,  then 
it's  as  good  as  never  being  done  in  the  first  place. 

DR.  REGAN:  As  formidable  as  Dr.  Rostker  is,  there  are  25  college 
students  standing  outside  this  door,  and  I guess  I'm  probably  even 
more  terrified  of  them.  I regret  very  much  that  Dr.  Rostker  will 
not  be  able  to  join  us  for  the  next  several  days,  because  I thought 
we  might  be  able  to  reschedule  the  program  to  accommodate  what  is 
a critical  discussion,  it  seems  to  me,  for  a variety  of  reasons. 

But  short  of  that,  perhaps  we  can  work  out  some  forum  for  continuing 
discussion,  which  I really  do  apologize  for  having  to  truncate  at 
this  time.  As  you  recall,  we  had  planned  to  have  Or.  Rostker's 
presentation  this  morning,  and  I felt  1 could  deal  somewhat  more 
easily  with  Admiral  Metcalf  this  evening.  But  I didn't  realize  that 
it  was  going  to  turn  out  this  way.  I think  you'll  all  agree  that  we 
have  heard  a provocative  set  of  remarks--that ' s in  the  form  of  a 
euphemism.  But  nobody  would  classify  it  as  entertainment,  1 guess. 
We're  going  to  have  some  now  by  a group  named  the  National  Review 
Troupe,  Mesa  College  students.  They've  performed  at  the  Veterans 
Administration,  the  Kiwanis  Club,  the  USD,  Children's  Hospital, 

City  College,  and  the  Psychiatry  Department  of  the  University  Hospital. 
At  that  location  they  confined  themselves  to  upbeat  and  happy  songs, 
and  they  told  me  that  that's  what  they're  going  to  do  tonight,  so 
you  can  draw  your  own  conclusion.  And  they're  going  to  come  on  now 
whether  we  want  it  or  not. 
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MILITARY  SERVICE  GROUP  MEETING:  PLENARY  SESSION 


SUMMARY 


In  this  portion  of  the  symposiwn  participants  from  each  branch  of  the 
service  met  in  separate  small  groups  (joined  by  faculty  members  and 
others)  to  identify  barriers  to  utilization  of  human  resources  RDT&E 
and  possible  strategies  for  improving  utilization.  The  groups  then 
reported  their  delidoerations  in  plenary  session. 

Dr.  Edward  Glaser,  the  faculty  chairman,  in  orienting  the  conferees 
to  the  task  of  the  small  groups,  suggested  that  they  consider  a se- 
ries of  issues  regarding  utilization  that  were  raised  at  a recent 
symposium  on  the  management  of  federal  BSD.  At  that  symposium,  it 
was  suggested  that  B&D  project  plans  should  be  required  to  address 
such  questions  as:  Who  are  the  potential  users?  How  might  they  be 
reached?  With  what  incentives  to  use  the  product?  Bow  might  tech- 
nical assistance  aid  the  adoption  process?  What  are  the  applications 
and  policy  implications  of  the  projected  findings  or  product?  How 
will  they  be  communicated  to  policymakers? 

In  the  plenary  session  following  the  small  group  deliberations  the 
chairman  of  the  Air  Force  Group,  Dr.  Herbert  Clark,  Air  Force  Human 
Resources  Laboratory  (HRL),  reported  a number  of  procedures  adopted 
by  HRL  to  promote  better  utilization.  These  included: 

. Collocation  of  researchers  and  customers. 

. Implementation  of  Air  Force  Regulation  80-51,  which  formalized 
the  process  by  which  customers  stated  R&D  requirements. 

. Initiation  of  "Trace" — a brief,  timely  summary  of  the  results 
of  an  R&D  project, 

. Establishment  of  an  applications  office  with  responsibility  for 
tracking  R&D  and  facilitating  customer  involvement. 

. Introduction  of  the  work  unit  review  cycle. 

The  Army  group,  chaired  by  Dr.  Arthur  Drucker,  Army  Research  Institute, 
addressed  the  question  of  how  to  get  sustained  sponsor  involvement — 
without  losing  control.  Their  suggestions  included: 

. Reach  agreement  in  the  initial  research  plan  as  to  when  the  appli- 
cation phase  will  be,  and  who  will  do  what  at  that  time. 

. Remind  the  sponsor  of  his  obligation  under  Army  Regulation  70-8 
to  report  on  the  utilization  of  end  products. 

. Make  the  customer  a believer  in  the  research  through  periodic 
formal  and  informal  reviews. 

. Get  the  military  to  sell  the  research  to  the  military. 

. Make  use  of  the  Technical  Advisory  Service, 
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Theve  weve  two  Navy  gvoups.  One,  ahaived  by  Dr.  Robert  Smith,  Office 
of  the  Chief  of  Naval  Operations,  considered  some  specific  recommenda- 
tions of  the  Defense  Science  Board  Task  Force  on  Training  Technology 
that  were  addressed  to  utilization.  This  group  noted: 

. The  need  of  the  R&D  producer  for  specific  guidance  as  to  product 
requirements. 

. The  need  for  cognizance  of  the  resource  programming  cycle. 

. The  need  for  more  direct  ways  of  communicating  the  fleet's  problems 
to  the  R&D  setting  and  R&D  results  to  the  fleet. 

. The  possibility  of  special  billets  within  the  fleet  to  assist  in 
implementation. 

. The-  need  for  joint  planning  for  utilization  by  R&D  users,  sponsors 
and  producers, 

. The  need  for  producers  and  operators  to  have  greater  familiarity 
with  each  others  communities . 

The  second  Navy  group,  chaired  by  Dr.  Glenn  Bryan,  Office  of  Naval 
\ Research,  reported  a nianber  of  suggestions  for  improving  the  utiliza- 

j . tion  of  people-related  R&D,  including: 

I 

, The  need  for  cross  education  of  researchers  and  users,  including 
an  intensive  process  of  getting  acquainted  and  the  need  for  re- 
quirements to  be  stated  in  language  that  researchers  can  under- 
stand and  reports  to  be  tailored  to  specific  users. 

. The  need  for  early  and  reiterated  cost-benefit  analysis  of  R&D 
products,  where  appropriate . 

The  need  for  formal  mechanisms  for  involving  users  with  researchers 
in  all  stages  of  the  research. 

Need  for  recognition  that  the  customer  for  some  (basic)  research 
is  another  (mission-oriented)  researcher. 

I The  moderator  of  the  plenary  session.  Dr.  Jay  Uhlaner,  Army  Research 

Institute,  contributed  several  observations: 

. No  single  utilization  strategy  will  do  the  job.  For  example, 
different  solutions  may  be  required  in  different  program  cate- 
gories (6.1,  6.2,  etc.)  and  for  different  users. 

. Utilization  is  part  of  program  development,  and  consequently 
solutions  must  take  cognizance  of  the  life-cycle  management  of 
the  total  program  development. 

. There  is  need  for  more  "horsepower"  at  various  levels  of  the  ap- 
plications engineering  effort. 

, There  is  need  for  procedures  by  which  a community  of  users  can 
! prioritize  their  requirements  so  the  research  community  can  carry 

1 out  a balanced  program  . 

' . DDR&E  should  be  apprised  regarding  reasonable  expectations  for 

payoff  in  the  various  program  categories  . 

.Strategies  are  needed  for  exploiting  the  specialized  competence  of 
each  player  (e.g.,  operator,  researcher,  rest  zrch  manager)  rather 
than  assigning  all  tasks  to  everyone. 
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GROUP  INTERACTION  GUIDANCE 


Dr.  Edward  M.  Glasev 
President 

Human  Interaction  Research  Institute 


In  planning  this  conference,  we  gave  a good  deal  of  thought  to  its 
length  since  one  doesn't  lightly  ask  people  to  take  off  3 1/2  days 
to  attend  a meeting.  The  reason  that  we  did  it  this  way  is  reflected 
in  the  opening  remarks  made  by  this  morning's  speakers.  Research 
utilization  is  a difficult  and  complex  problem,  beset  by  many  ob- 
stacles and  it  will  take  time  to  reduce,  let  alone  resolve  them. 

Last  week  in  Washington,  there  was  a two-day  symposium  on  the  manage- 
ment of  federal  R60.  Glenn  Bryan  was  there,  as  was  I.  Glenn  made 
an  important  point  that  has  not  been  mentioned  this  morning.  He 
stated  that  before  you  get  to  the  stage  of  investing  a good  deal  of 
time,  energy  and  money  in  promoting  the  utilization  of  any  R6D  out- 
come, you  should  cross-va 1 1 date  it  and  determine  if  it  truly  works 
under  operational  conditions. 

Now,  considering  the  fact  that  research  by  its  nature  is  something 
of  a risky  business  and  there  is  no  guarantee  that  every  hole  you 
drill,  so  to  speak,  will  produce  a gusher,  there  is  something  that 
follows  from  that  in  relation  to  the  GAO  report.  As  I recall  the  GAO 
figures,  they  indicate  that  while  quite  a few  of  the  projects  have 
not  been  used,  according  to  their  criteria,  56  percent  have  been  used. 
The  fact  that  a majority  of  the  projects  did,  according  to  their 
investigation,  find  application  seems  to  me  a rather  commendatory 
finding.  That  doesn't  change  the  fact  that  they  found  38  percent 
that  have  not  been  used.  That  figure  can  be  reduced  significantly  by 
some  of  the  activities  that  Admiral  Metcalf  and  Colonel  Taylor  suggest- 
ed this  morning  in  terms  of  early  relationship  with  the  user. 

At  the  symposium  last  week,  one  of  the  points  made  that  might  be 
relevant  for  your  consideration  at  this  conference  was  that  utilization 
would  be  facilitated  if  more  R&D  funding  sources  required  applicants 
to  think  about  the  following  sorts  of  questions  in  preparing  research 
proposal s ; 

1.  Who  is  going  to  care  about  what  has  been  found  out  or  developed? 

2.  What  categories  of  institutions,  groups,  or  persons  would 
constitute  the  audiences,  beneficiaries  and  potential  users 
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of  the  findings  if  they  turn  out  to  be  valuable  and  ready  for 
d i ssemi nat ion? 


How  might  these  potential  users  be  reached? 

4.  What  incentives  or  procedures  might  be  employed  to  interest 
them  in  wanting  to  try  the  innovation? 

5.  If  an  offer  of  technical  assistance  is  desirable  to  facilitate 
the  transfer  or  adoption  process,  how  might  that  be  provided? 

6.  How  do  anticipated  findings  of  the  project  relate  to  service 
delivery  and  support  systems,  given  program  efforts,  teaching, 
training  or  staffing  efforts,  other  R&D  activities? 

7.  Might  the  findings  from  this  project  have  policy  or  legislative 
implications?  If  so,  for  whom?  At  what  levels?  What  sorts  of 
i mpl i cations? 

8.  And  what  is  the  plan  for  bringing  the  findings  to  the  attention 
of  various  relevant  categories  of  policy  or  political  decision- 
makers? 

When  you  break  into  small  groups  this  afternoon,  you  might  find  it 
relevant  to  consider  questions  like  those  just  enumerated  in  addition 
to  the  ones  that  have  been  included  in  the  boxed-off  material  on 
pages  10  and  11  of  your  conference  agenda. 

The  rationale  for  breaking  into  small  groups  is  twofold.  One  is  that 
at  a conference  where  a major  part  of  the  time  is  spent  listening  to 
talks,  however  interesting  and  stimulating,  audience  participation 
is  limited.  You  are  very  knowledgeable  individuals  who  have  many 
experiences  and  perceptions  to  share  with  each  other.  Small  groups 
promote  the  active  participation  which  we  need.  The  other  is  that 
the  Army,  Navy,  Air  Force  and  Marine  Corps  have  unique  problems  in 
addition  to  their  common  problems,  and  thus,  each  service  needs 
time  to  identify  its  own  problems  and  its  own  proposed  strategies  for 
cop i ng  wi th  them. 


AIR  FORCE 


Dr.  Herbert  J.  Clark 
Chief  of  Plans  and  Programs 
Air  Force  Human  Resources  Laboratory 


We  had  a rather  interesting  discussion  about  the  different  procedures 
that  the  Air  Force  has  adopted  to  encourage  better  utilization  of  R6D. 
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It  may  be  useful  for  me  to  discuss  some  of  these.  I am  speaking 
primarily  of  the  human  resources  program  of  the  Human  Resources  Labora- 
tory rather  than  the  Air  Force  as  a whole. 

The  Human  Resources  Laboratory  (HRL)  was  established  in  I968.  There 
was  concern  at  that  time  that  R£D  products  in  the  human  resources 
area  were  not  being  implemented  adequately.  So  one  of  the  first 
decisions  was  that  the  researchers  should  be  collocated  with  the 
user.  Instead  of  continuing  to  conduct  our  R&D  and  flying  training 
and  technical  training  at  Patterson  Air  Force  Base,  we  established 
two  divisions,  one  at  Phoenix  and  one  at  Lowry  Air  Force  Base  for 
flying  and  technical  training,  respectively. 

Collocation  proved  to  have  many  advantages.  Our  divisions  can  work 
directly  with  the  customer;  we  can  use  Air  Force  trainees  as  research 
subjects;  and  we  can  communicate  much  better  with  the  customer. 

There  are  also  disadvantages  as  far  as  HRL  headquarters  is  concerned. 
The  distance  creates  management  problems;  there  is  a tendency  for 
the  division  to  establish  an  extremely  strong  relationship  with 
the  customer,  and  sometimes  become  more  responsive  to  the  customer 
than  to  headquarters. 

Another  procedure  that  we  implemented  in  the  laboratory  to  ensure 
good  communication  and  contact  with  the  customer  was  Air  Force 
Regulation  8O-5I.  Basically,  this  requires  the  customer  to  submit 
a formal  request  to  the  laboratory  when  he  wants  assistance  in  R&D 
in  personnel  selection  and  classification  or  flying  or  technical 
training  technology.  The  request  does  not  come  directly  to  the 
laboratory;  it  goes  to  the  Air  Staff  and  the  Systems  Command  and, 
at  the  same  time,  we  make  a technical  evaluation  of  that  request. 

If  it  is  within  our  resources  and  capability  to  conduct  the  R&D, 
we  will  carry  it  out.  The  end  result  is  a formal  relationship  with 
the  customer.  There  are  some  pros  and  cons  to  this,  and  I would  like 
to  come  back  to  them  in  a moment. 

In  working  with  the  customers,  we  have  had  numerous  complaints  that 
the  results  of  R&D  projects  are  not  received  soon  enough  and  are 
poorly  communicated.  So  we  have  established  something  called  a 
"Trace,”  which  is  a quick  summary  in  layman's  terms  of  the  results 
of  the  study.  Sometimes  this  is  distributed  several  months  before 
the  technical  report  is  published,  and  deals  quite  well  with  the 
problem  of  providing  the  customer  with  something  formal.  It  is  also 
distributed  to  other  interested  parties  both  inside  and  outside 
of  the  Ai r Force . 

In  addition,  we  have  established  a separate  applications  office 
(I  think  the  Army  and  Navy  already  have  a number  of  these)  which  has 
the  responsibility  of  tracking  the  R&D  to  guarantee  that  the  customer 
understands  it  and  knows  what  steps  to  take  to  implement  it,  if 
this  is  appropriate,  and  to  do  cost  analysis  both  at  the  beginning 
and  end  of  a research  project. 
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We  have  also  introduced  a work  unit  review  cycle,  which  requires  that 
all  work  units  in  the  laboratory  be  reviewed  at  least  twice  a year 
and  that  at  the  time  each  work  unit  is  established  the  benefit  to 
the  customer  must  be  identified,  and  an  estimate  of  return  on 
investment  must  be  made. 

We  have  three  R's  in  the  laboratory  for  establishing  priorities  and 
implementing  research  programs:  a requ i rement  wh i ch  can  either  be 
from  a customer  or  in-house,  the  available  resources  to  support  this 
requirement,  and  the  expected  resul ts . If  the  anticipated  results 
are  reasonable  both  in  the  eyes  of  the  laboratory  and  the  customer, 
we  will  undertake  the  project.  However,  we  are  insisting  more  and 
more  that  the  customer  understand  and  support  what  we  are  doing. 
Otherwise,  we  have  found  that  he  may  be  disappointed  with  the  results 
and  refuse  to  implement  them. 

In  the  group  we  discussed  the  pros  and  cons  of  having  a formal 
procedure  for  ensuring  implementation  of  research  products.  At  HRL 
we  have  tried  to  avoid  the  formality  and  depend  on  the  goodwill 
of  both  the  customer  and  the  laboratory,  but  it  has  not  worked  out 
very  well.  The  group  felt  that  it  is  necessary  to  have  a formal 
regulation  which  requires  a written  agreement  between  the  laboratory 
and  the  customer  with  reference  to  each  research  project.  This  raises 
some  issues,  however.  For  example: 

1.  Since  the  customer  submits  requirements  throughout  the  year, 
and  we  respond  to  them,  what  happens  if  we  run  out  of  money? 

2.  Should  these  requirements  be  funded  individually  or  should 
we  take  the  money  out  of  one  big  pot?  If  the  money  comes 
out  of  one  big  pot,  we  may  find  out  that  at  the  end  of  the 
fiscal  year  we  are  out  of  money  and  can't  satisfy  the  re- 
qu i rements . 

3.  A limitation  in  the  regulation  is  that  all  of  the  requirement 
for  communication  seems  to  be  on  the  laboratory;  e.g.,  to 
provide  project  results  to  the  customer.  Now  the  regulation  is 
being  revised  to  require  that  the  customer,  upon  receipt  of  the 
results,  give  feedback  to  the  laboratory  through  our  higher 
headquarters,  indicating  whether  they  are  satisfied  or  dis- 
satisfied. I think  this  was  brought  about  by  the  GAO  study 
and  we  hope  that  in  the  future  we  will  be  in  a better  position 
to  know  what  proportion  of  our  R&D  has  been  implemented. 

h.  We  discussed  the  question  of  what  criteria  are  appropriate  for 
determining  whether  or  not  a product  has  been  implemented,  but 
we  didn't  reach  any  firm  conclusion  on  that.  I think  that 
is  an  extremely  difficult  question  to  answer  and  probably 
requires  a symposium  in  itself. 

In  terms  of  recommendations  for  the  symposium,  we  envision  that  what 
we  are  talking  about  today  is  simply  food  for  thought,  and  that  as 
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time  goes  on,  we ' I I be  able  to  develop  some  specific  recommendat i ons . 
Perhaps  out  of  this  symposium  can  come  recommendations  that,  after 
appropriate  service  staffing,  could  be  forwarded  to  DORSE;  for  example, 
for  their  consideration.  It  would  be  appropriate  to  have  this  meeting 
produce  a concrete  product  that  can  be  utilized  rather  than  simply  a 
reiteration  of  issues  that  I'm  sure  we  all  have  discussed  in  the  past 
and  will  continue  to  discuss  in  the  future. 

What  I have  tried  to  pass  on  to  you  today  is  simply  some  of  the 
procedures  that  the  Air  Force  uses  for  carrying  on  its  business  of 
i mp  1 er.ien  t i ng  its  R6D  products,  and  some  of  the  perils  involved  in 
the  techniques  that  we  have  developed. 

DISCUSSION 

QUESTION;  Do  you  feel  satisfied  with  your  current  procedures  for 
achieving  involvement  with  users? 

DR.  CLARK:  1 would  say  that  1 am  quite  satisfied  with  the  procedures. 
In  the  past  we  would  often  get  an  informal  request  from  a customer, 
who  might  be  a captain  in  the  Air  Training  Command,  for  a particular 
research  activity,  and  nobody  has  ever  heard  of  the  requirement.  As  it 
stands  now,  our  formal  requirement  comes  from  the  Systems  Command  and 
is  paralleled  by  a requirement  in  the  Air  Training  Command.  They  now 
have  a research  review  board.  Before  they  send  a requirement  to  our 
laboratory,  they  staff  it  so  that  Air  Training  Command  is  sending  the 
request.  There  is  a formal  record  of  this;  if  the  captain  leaves, 
it  doesn't  matter.  And  we  can  go  back  to  them  and  brief  them  periodi- 
cally; they  know  what  the  status  is.  And  I think  that  has  worked  out 
well. 

In  addition,  in  some  cases  we  have  been  a little  short  of  funds  and 
we  have  found  that  at  times  the  customer  has  come  to  our  rescue  by 
providing  travel  funds  or  letters  of  support  to  our  headquarters  to 
ask  that  funds  be  reinstated  to  our  laboratory  for  support  of  their 
programs.  1 think  that  there  are  some  problems,  though.  One  of  the 
major  problems  is  that  as  you  get  involved  in  this  sort  of  applications 
orientation,  the  personnel  have  to  come  from  somewhere,  time  has  to 
come  from  somewhere,  and  that  comes  from  the  R&D  program  itself.  And 
this  is  of  some  concern.  It  is  also  of  concern  to  the  degree  that 
the  customer  is  sometimes  a little  shortsighted  and  is  only  looking 
at  an  immediate  problem.  We  feel  that  we  have  to  keep  a balance 
between  the  technology  base  and  the  technology  applications  program, 
but  requests  from  the  customers  are  generally  short-term,  high-payoff, 
simple  problems,  perhaps  just  technology  application.  This  tends  to 
drive  the  program,  and  1 view  that  with  considerable  concern,  because 
1 don't  think  we  should  become  so  deeply  involved  with  the  customer 
that  we  don't  have  a technology  base  program.  As  it  stands  now, 
about  50  percent  of  our  program  is  oriented  toward  response  to 
customer  requirements.  That  may  involve  technology  base  requirements 
too,  but  typically  does  not.  The  other  50  percent  is  sel f- i n i t i ated 
in  response  to  a requirement  or  need  that  we  sense. 


-1*3- 


QUESTION:  Does  the  Air  Force  cover  that  side,  the  se I f- i n i t iated  side? 
How  does  it  get  formally  processed  through  the  channels? 


DR.  CLARK:  There  is  a procedure  in  the  Systems  Command,  particularly 
in  regard  to  the  6.3  program,  it  is  called  the  investment  strategy 
procedure.  For  each  of  our  6.3  programs,  we  prepare  a technology 
program  plan,  which  we  send  annually  to  the  potential  customers.  They 
make  an  evaluation  in  accordance  with  the  particular  format  described 
in  the  regulations.  They  send  this  evaluation  to  our  higher  head- 
quarters, saying,  "This  is  a good  project;  we  support  it."  They  grade 
it,  they  rank  it,  and  so  forth,  and  that  has  considerable  impact  on 
whether  or  not  that  6.3  project  will  be  funded. 

The  6.1  program  in  the  Air  Force  is  consolidated  in  one  organization, 
AFOSR,  who  controls  the  funds.  We  tell  them  what  we  propose  to  do, 
and  if  they  like  it,  it  is  their  responsibility  (along  with  ours)  to 
ensure  that  the  customers  like  it  too.  But  if  they  don't  like  what  we 
are  doing  in  6.1,  they  can  reorient  us;  they  can  change  our  funds. 

In  the  6.2  area,  the  primary  controlling  mechanism  is  the  work  unit 
review.  Each  new  work  unit  is  required  to  establish  schedules  for 
the  completion  of  that  work.  The  initial  work  unit  review  is  con- 
ducted when  the  work  is  started  and  is  kept  in  the  file.  Then,  twice 
a year  there  is  a periodic  work  unit  review  (once  by  the  division, 
once  by  the  headquarters),  to  determine  whether  you  are  on  schedule 
and  whether  the  product  that  you  are  planning  will,  in  fact,  occur 
at  the  time  that  you  are  expecting  it  to.  But  not  every  6.2  program 
has  a product.  I think  we've  talked  about  that  this  morning.  I 
don’t  think  everything  we  do  should  necessarily  have  a customer  in 
mi  nd . 

QUESTION:  Where  is  the  balance  between  6.1,  6.2,  6.3,  and  6.^4  phases 
of  the  total  program?  Where  is  that  decision  made? 

DR.  CLARK:  In  Congress.  As  a matter  of  fact,  our  program  was  just 
cut . 

QUESTION:  I mean  before  that;  how  are  decisions  made  about  what  the 

Air  Force  presents  to  the  Congress? 

DR.  CLARK:  The  balance  between  6.1,  6.2,  and  6.3  within  the  labora- 
tories has  been  somewhat  the  same  over  the  years  and  it  doesn't  vary 
appreciably,  no  matter  what  you  might  submit  in  terms  of  justification 
for  additional  funds. 

QUESTION:  Since  you  have  adopted  the  formal  procedure,  what  has 
been  the  demand  from  the  user  community?  Is  it  Increasing,  or  are 
you  able  to  hold  it  in  check,  or  are  you  able  to  maintain  this  kind 
of  50-50  balance  with  the  technology  base  program,  or  is  it  an  in- 
creasing function? 

DR.  CLARK:  It  turns  out  that  we  complete  RPRs  (user  requests)  at  a 
rate  that  has  enabled  us  to  maintain  the  50-50  ratio  with  se 1 f- i n i t i ated 
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research.  On  occasion,  we  have  had  to  defer  an  RPR  until  the  next 
fiscal  year.  If  we  have  to  have  contract  funds  for  it,  we  will  need 
to  defer  it  until  the  next  year. 

QUESTION:  I have  a question  on  who  prepares  the  cost  benefit 

analysis.  Presumably  it  would  be  the  customer  if  he  initiated  it, 
but  if  somebody  within  your  laboratory  wants  to  do  it  and  goes  in 
search  of  the  customer,  does  the  researcher  prepare  the  cost-benefit 
analysis  or  does  the  customer  do  it? 

DR.  CLARK:  At  this  point,  we  haven't  asked  the  customer  to  do  a 
cost-benefit  analysis,  and,  in  fact,  he  has  often  asked  us  to  do 
it.  Although  our  new  applications  office  is  small,  we  do  have  a 
cost-benefit  analysis  capability  in  that  office,  and  on  selected 
programs  we  do  ask  what  the  payoff  would  be.  If  it  is  a simulator 
program,  for  example,  we  would  I i Ke  to  know  how  much  that  simulator 
is  going  to  cost,  when  it  is  going  to  amortize  itself,  and  what  the 
difference  is  between  the  simulator  and  flying  the  airplane. 

DR.  UHLANER:  I think  we  might  now  ask  Dr.  Drucker  to  present  the 
report  from  the  Army  group. 


ARMY 


Dr.  Arthur  J.  Drucker 
Chief,  Plans  and  Operations 
Army  Research  Institute 
for  Behavioral  & Social  Science 


Our  group,  which  essentially  consisted  of  members  of  the  Army 
Research  Institute  (ARl)  and  the  Human  Engineering  Laboratories 
(HEL)  , talked  of  many  things.  We  were  assisted  greatly  by  some 
walk-in  faculty  members  who  were  most  helpful  in  stimulating  the 
d i scuss i on . 

Although  we  talked  for  about  2 1/2  hours,  my  presentation  is  very 
brief.  It  is  a distillation  of  what  we  regarded  as  a major  problem 
in  ut i 1 i zat ion--how  to  get  sponsor  involvement  and  keep  it  throughout 
the  research  and  into  the  utilization  phase.  We  decided  that  if 
we  could  get  the  sponsor's  involvement  at  the  very  beginning,  if 
he  was  sincere  about  the  requirement,  if  he  was  still  around  during 
and  following  the  research,  and  if  the  situation  giving  rise  to  the 
requirement  had  not  changed,  we  would  be  in  pretty  good  shape. 

But  how  do  we  get  his  involvement  without  losing  control?  This 
has  been  a very  real  problem.  We  decided,  without  exploiting  our 
experience  as  clinical  psychologists,  that  this  was  perhaps  a matter 
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of  personalities.  If  we  happened  to  get  the  right  kind  of  sponsor, 
we  were  in  pretty  good  shape.  But  that  was  a risk.  We  have  had 
too  many  instances  of  the  camel's  nose  in  the  tent  and  yet  we  have 
faced  up  to  that  problem  where  we  had,  indeed,  been  in  bed  with  the 
sponsor  and  his  representative,  and  managed  to  stay  alive  resea rchwi se . 

Following  are  some  of  our  suggestions  for  stimulating  sponsor  in- 
volvement without  losing  control: 

1.  Try  to  get  an  agreement,  in  the  original  research  plan,  as 

to  when  the  utilization  phase  will  be  and  who  will  do  what  at 
that  time. 

2.  If  necessary,  lean  on  Army  regulation  70-8,  which  goes  into 
considerable  detail  concerning  the  establishment  of  requirements 
and  the  building  of  the  program.  It  includes  two  sections  on 
what  the  sponsor  is  supposed  to  do,  by  regulation.  One  section 
of  this  regulation  requires  the  sponsor  to  report  on  the 
utilization  of  end  products  when  the  research  is  completed. 

It  requires  statements  of  how  it  was  utilized,  what  his 
reactions  were  to  the  research,  and  what  changes  he  would  have 
preferred  in  the  way  the  project  was  carried  out. 

3.  If  he  is  not  already  a believer  in  the  research,  make  him  one, 
even  if  you  have  to  resort  to  the  IPR  (in  Process  Review)  which 
is  part  of  regulation  70-8,  or  preferably  an  ipr  (informal 
periodic  review).  In  our  experience,  this  has  been  more  satis- 
factory, since  we  can  get  down  to  the  task  unit  level  and 
really  work  with  some  of  the  sponsor  representatives  in  con- 
siderable detail  and  obtain  their  cooperation. 

h.  Get  the  military  to  sell  the  product  to  the  military  when  the 
utilization  stage  is  near.  This  has  worked  out  well  for  the 
Army  both  at  ARI  and  HEL  (e.g.,  ARI's  effort  with  computer-based 
training,  the  skill  qualification  test,  REALTRAIN  and  HEL's 
hea 1 th  act) . 

Another  form  of  utilization  that  we  have  perhaps  not  given  much 
attention  to  is  TAS  (Technical  Advisory  Service)  as  it  is  practiced 
in  ARI.  TAS  is  particularlv  important  in  the  10  field  units  which 
we  have  working  within  Army  installations.  Through  TAS  we  help  pay 
our  rent,  so  to  speak,  to  the  installation.  We  provide  services 
to  the  installation  commander;  we  gain  acceptability;  and  we  advise 
him  on  the  proposals  that  he  may  want  to  consider  (e.g.,  don't  spend 
money  on  this;  20  years  ago  we  researched  that  problem  and  we  are 
pretty  sure  the  answers  we  derived  then  are  the  answers  today). 

You  know  that  you  have  really  made  it  with  the  sponsor  when,  at 
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the  conclusion  of  a piece  of  research,  he  tries  to  give  you  O&M  money 
that  he  wants  you  to  use  for  what  would  pass  as  a 6.4  utilization 
J effort.  Should  we  encourage  him  to  do  that?  Well,  sometimes  we 

t do  and  sometimes  we  don't.  I say  that  with  6.4  type  effort,  perhaps 


it  is  O&M  money  that  he  wants  to  pass  to  us.  I'm  not  quite  sure 


what  I'd  recommend-- to  accept  or  not.  I'll  leave  that  to  my  boss 
here  (Dr.  Jay  Uhlaner). 


It  was  also  pointed  out  in  the  discussion  that  there  probably  is 
more  utilization  of  our  products  than  we  will  ever  know.  For 
example,  ARI  puts  out  about  a thousand  copies  of  every  technical 
publication;  these  and  other  products  get  into  the  public  domain. 

The  GAO  hit  us  on  this,  and  one  of  our  resolves  in  the  group 

was  that  we  certainly  should  keep  better  records  of  where  our  products 

go,  who  gets  them,  and  where  we  should  try  to  gain  credit. 


We  then  discussed  one  of  the  questions  on  the  agenda;  "Is  there 
significant  opportunity  for  enhancing  the  transfer  of  certain  mili- 
tary RDT&E  findings  to  the  private  sector?"  We  feel  that  we  have  a 
long  record  of  transmitting  such  products  in  the  form  of  publications 
and  other  nonpublication  products  to  the  public  sector.  I could 
talk  at  considerable  length  about  selection  tests  that  have  been 
transmitted  to  the  private  sector,  such  as  state  and  city  governments, 
even  some  foreign  governments  when  we  are  permitted  to  do  so.  In  the 
release  process  to  the  public  sector,  we  have  had  incidents  of  test 
compromise  where  there  was  need  to  safeguard  tests  and  products  of 
that  sort  against  such  compromise. 

Finally,  let  us  talk  very  briefly  about  Question  6-B:  "What  long- 
run  changes  are  necessary  to  improve  utilization?"  If  we  ever  go 
to  a formal  6.4  program,  it  is  our  feeling  that  we  should  try  to 
staff  such  an  organization,  if  we  could  afford  it,  with  middle-level 
scientific  talent.  It  is  our  considered  opinion  that  we  do  not  need 
the  very  high-level  talent,  except  for  the  directorship  of  such 
efforts;  GS-lls  and  12s  might  very  well  suffice  for  that  kind  of 
operation  should  we  ever  get  to  that  point. 


DISCUSSION 

QUESTION:  I understood  you  to  say  that  ARI  publishes  and  distributes 

research  reports.  What  I didn't  understand  was  something  you  said 
right  after  that  to  the  effect  that  because  of  GAO's  comments  you 
would  have  to  get  your  house  in  order  and  know  better  who  gets  them. 

I don't  understand  how  you  distribute  something  without  knowing  who 
gets  it. 

DR.  DRUCKER:  Well,  as  far  as  original  distribution,  we  do  know  who 
receives  the  reports.  In  my  remark  I was  reflecting  the  question 
regarding  which  of  those  thousand  recipients  were  the  ones  who  shoula 
not  only  have  received  it  but  also  owned  up  to  it  when  GAO  came 
around  to  check  up.  I think  we  all  had  that  kind  of  difficulty  and 
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frequently  when  GAO  went  to  the  designated  sponsor  or  user,  they  drew 
a blank;  that  is,  the  person  they  checked  with  in  that  command  did  not 
recall  receiving  the  report. 

DR.  UHLANER:  Let  me  elaborate  on  that  for  just  a moment.  We  sent 
it  out  on  the  standard  distribution  but  GAO  randomly  chose  recipients 
to  check  with.  Some  of  them  said,  "I  never  saw  it."  It  was  addressed 
proper'y,  but  for  all  we  know,  it  might  have  ended  up  in  the  library 
or  with  an  assistant  or  somewhere  else.  1 might  point  out  that  GAO 
used  us  as  sort  of  the  initial  training  ground,  and  so  some  of  the 
things  that  happened  later,  we  hope,  got  ironed  out  after  they  were 
through  wi th  us . 

QUESTION:  Why  do  you  assume  that  the  formal  6.^  program  is  such 
that  a lower  level  of  technical  people  can  be  used? 

DR.  DRUCKER:  The  specific  reasoning  behind  that  remark  was  that  less 
creativity  or  scientific  talent  seems  required  for  application.  That 
was  one  reason.  Another  was  that  many  bodies  would  undoubtedly  be 
required,  so  there  is  a matter  of  pure  economics. 

QUESTION:  It  seems  that  what  you  are  saying  in  the  attitudes  you 

are  representing  is  that  application  research  is  beneath  the  dignity 
of  the  researcher,  and,  by  God,  he  shouldn't  get  involved  in  6.^. 

DR.  DRUCKER:  Well,  I don't  think  we  intended  that.  It  is  a point  well 
taken,  and  I agree  that  that  is  a bad  and  erroneous  impression. 

QUESTION:  You  really  need  to  think  about  a range  of  levels  within 
each  of  the  6.1,  6.2,  6.3  and  6. A levels;  in  fact,  a lot  of  research 
can  be  done  by  people  on  all  those  levels.  But  you  seem  to  assume 
that  translation  or  transformation  functions  are  at  a much  higher 
order  of  skill  than  application  functions.  One  of  the  problems  is 
that  you  did  fail  to  recognize  the  complexities  and  skill  requirements 
for  effective  application. 

DR.  UHLANER:  Vie' 11  come  back  to  that  point,  but  I think  I'm  going 
to  postpone  that  until  we  hear  the  other  discussions.  Let's  move 
onto  the  Navy,  under  the  chairmanship  of  Bob  Smith. 

NAVY,  GROUP  1 


Df.  Robert  G.  Smith,  Jr. 
Assistant  for  Training 
and  Personnel  Technology  Planning 
R&D  Plans  Division 

Office  of  the  Chief  of  Naval  Operations 


I think  it  is  important  in  light  of  the  preceding  discussion  that  I 
sketch  some  of  the  major  managerial  outlines  of  the  Navy  R&D  program, 
as  a context  for  things  that  will  be  said  later. 
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Within  the  Navy,  the  Assistant  Secretary  of  the  Navy  (Research, 
Engineering  and  Systems)  has  control  over  RsD  appropriations,  and  he 
delegates  responsibility  for  RDTSE  management  to  different  parts  of 
the  Navy.  The  6.1  (Research)  responsibility  goes  to  the  Chief  of 
Naval  Research.  The  6.2  (Exploratory  Development)  responsibility 
goes  to  the  Chief  of  Naval  Development,  who  also  holds  a position  in 
the  Naval  Material  Command.  In  general,  the  work  there  is  planned 
through  a series  of  strategies,  and  Jim  Regan,  the  Technical  Director 
of  NPRDC,  is  the  principal  strategist  for  the  training  and  personnel 
area.  The  control  of  6.3  (Advanced  Development)  and  6. A (Engineering 
Development)  work  is  delegated  to  the  Office  of  the  Chief  of  Naval 
Operations.  The  Director  of  RDTf.E  (OP-098)  implements  the  responsi- 
bilities of  the  CNO  in  this  area.  His  office  develops  and  staffs 
requirements  for  the  6.1  and  6.2  work,  and  operational  requirements 
for  6.3  and  6. A work.  There  is  essentially  a matrix  organization  of 
6.3  and  6. A activities;  someone  from  the  Director  of  RDTEE's  office 
is  paired  with  a sponsor  representative  t.  jm  the  appropriate  office 
within  the  CNO's  staff.  (l,  for  example,  am  paired  with  Merle 
Malehorn,  representing  the  Director  of  Naval  Education  and  Training 
in  training  and  education  work,  and  Mike  Letsky,  representing  the 
DCNO  (Manpower)  in  manpower  and  personnel  work).  This  is  an  outline 
of  how  the  Navy  system  works. 

The  group  that  I chaired  was  supposed  to  consider  some  specific 
recommendations  of  the  Defense  Science  Board  Task  Force  on  Training 
Technology,  especially  those  that  dealt  with  the  utilization  of 
training  technology  R6D.  We  broke  up  into  groups,  and  each  one 
addressed  a specific  issue. 

The  group  on  training  devices  reported  that  with  regard  to  application 
programs,  some  parts  of  the  Navy  were  better  thari  others  in  seeing 
that  6.2  and  6.3  work  in  training  device  technology  was  brought  to 
bear  on  the  actual  production  of  such  devices.  Their  recommendation 
was  that  we  try  to  see  if  we  could  bring  the  other  groups  up  to  the 
same  high  level  of  this  one.  On  the  research  side,  the  R&D  producer 
felt  that  he  needed  guidance  on  what  kinds  of  specific  products  to 
produce  in  order  to  support  utilization. 

The  group  that  v/as  concerned  with  technical  training  developed  the 
outline  of  an  important  model  for  implementation  beginning  with 
interaction  on  requirements,  going  through  the  R6D  process  with 
parallel  activities  between  the  R6D  producers  and  users,  and  winding 
up  with  the  actual  planning  and  implementation  of  the  work.  They 
were  especially  concerned  that  it  takes  roughly  three  years  to  plan 
the  resources  for  utilization,  and  so  were  trying  to  see  how  the  RSD 
and  planning  activities  could  be  meshed. 

The  group  that  was  concerned  with  fleet  training  felt  that  improve- 
ments needed  to  be  made  in  a variety  of  ways  and  that  better  ways  of 
communicating  results  to  the  fleet  and  more  direct  ways  of  getting 
the  fleet's  problems  back  into  the  R&D  setting  need  to  be  developed. 
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There  was  a suggestion  that  the  development  of  implementation  billets 
in  various  places  within  the  fleet  be  considered  so  that  assistance 
in  implementation  can  be  offered. 

One  Navy  agency  that  plays  an  important  role  in  supporting  the  Chief 
of  Naval  Education  and  Training  with  immediate  short-term  analyses  and 
evaluations  is  the  Training  Analysis  and  Evaluation  Group  directed  by 
A1  Smode  at  Orlando.  For  some  time,  I have  felt  that  it  would  be 
extremely  useful  to  bring  together  NPRDC  and  TAEG  so  that,  on  selected 
work,  TAEG  could  serve  as  a bridge  between  the  R&D  community  and  the 
school  community.  There  has  been  a preliminary  general  agreement 
that  this  will  be  tried  by  the  end  of  the  year;  there  should  be  a 
plan  developed  with  some  specific  provisions  for  this  bridging 
funct i on . 

Another  committee  was  asked  to  deal  with  the  subject  of  linkages  be- 
tween sponsors,  users,  and  R&D  producers.  They  noted  with  considerable 
interest  that  the  Chief  of  Naval  Education  and  Training  recently  met  in 
Memphis  to  discuss  Joint  planning  for  utilization  by  R&D  users,  sponsors 
and  producers.  Hopefully,  this  will  result  in  some  specific  actions 
for  the  1980  POM  and  budget.  That  activity  starts  this  fall.  This 
group  also  pointed  out  that  very  often  the  scientist  is  in  a position 
of  wanting  to  see  his  work  used,  but  at  the  same  time,  is  not  sufficient- 
ly familiar  with  all  the  details  of  the  planning  processes  that  are 
required  to  accomplish  this.  There  was  a feeling  that  the  scientist 
needs  to  get  a broader  and  more  thorough  knowledge  of  the  Navy  and  how 
it  works,  especially  that  part  of  the  Navy  toward  which  his  work  is 
aimed,  and  of  what  is  needed  to  Implement  research  results.  Those 
are  the  principal  points  discussed  by  my  subcommittee. 

DISCUSSION: 

QUESTION:  I would  Just  like  to  supplement  what  Bob  said  about  the  last 

working  group,  of  which  I was  a member.  We  also  felt  that  the  oper- 
ational military  man,  in  general,  needed  to  be  familiar  with  the  whole 
R&D  enterpr i se--who  is  watching  what  and  the  system  the  research  is 
worked  under,  the  constraints  and  the  limitations.  That  does  not  seem 
to  take  place  in  the  Navy;  I wonder  whether  it  takes  place  in  the 
Air  Force  and  the  Army;  do  they  get  this  at  the  War  College  academies? 

DR.  UHLANER:  I wi 1 1 comment  for  the  Army.  My  impression  is  that  you 
almost  have  to  be  an  RSD  officer  to  really  understand  the  R&D  system 
in  the  Army,  so  that,  for  example,  the  typical  best-qualified  officer 
is  usually  not  exposed  to  the  workings  of  the  R&D  process.  It  is 
not  his  fault;  he  is  Just  not  exposed  to  it.  I don't  know  about  the 
Ai r Force . 

DR.  CLARK:  I think  for  the  most  part  the  people  in  our  chain  are 
well  versed  in  R&D  procedures  throughout  the  system.  I am  thinking 
of  the  people  who  represent  us  at  higher  headquarters,  the  military 
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officers.  I think,  in  general,  they  are  well  selected  and  understand 
the  procedures  but,  certainly,  they  are  not  as  emotionally  involved  in 
the  methods  of  R&D  as  the  research  scientist  because  they  have  not 
worked  at  that  level. 

QUESTION;  How  are  fleet  requirements  communicated  to  operational 
decision  makers  in  the  Navy,  and  then  to  the  R&D  community? 

DR.  SMITH:  There  are  several  ways  whereby  fleet  requirements  are 
communicated  to  the  higher  reaches  of  the  Navy.  One  is  through 
various  fleet  readiness  reports  in  which  a variety  of  specific  problem 
areas  that  affect  readiness  are  cited.  These  include  mostly  short- 
term problems,  but  if  you  see  the  same  things  cropping  up  each  quarter, 
with  a little  imagination  you  can  often  convert  some  of  them  into  R6D 
requirements.  The  fleet  clearly  can  pose  formal  operational  require- 
ments; for  instance,  very  recently  we  were  stimulated  to  produce  a 
whole  range  of  operational  requirements  because  of  the  fact  that  the 
fleet  is  suffering  from  a lot  of  objection  to  its  use  of  certain 
land  for  bombing  and  gunnery  ranges.  And  so  this  stimulated  us  to 
generate  requirements  aimed  at  reducing  noise  and  destructiveness  of 
weapons . 

DR.  UHLANER:  The  last  working  group  was  chaired  by  Dr.  Glenn  Bryan. 

I think  he  is  well  known  to  all  of  you  so  I won't  take  any  more  time 
other  than  to  give  his  current  title  which  is  Director  of  the 
Psychological  Sciences  Division,  Office  of  Naval  Research. 


NAVY,  GROUP  2 


Ur.  Glenn  L,  Bryan 

Director,  Psychological  Sciences  Division 
Office  of  Naval  Research 


I would  like  to  elaborate  briefly  on  Bob  Smith's  comment  about  how 
6.1  money  is  managed  in  the  Navy.  It  is  given  to  the  Chief  of  Naval 
Research,  who  has  several  responsibilities,  one  of  which  is  to  serve 
as  the  senior  officer  in  charge  of  the  Office  of  Naval  Research. 
Actually,  he  is  in  charge  of  the  whole  6.1  appropriation,  wherever 
it  is  spent  in  the  Navy.  In  the  military  science  program,  that  is 
$178,000,000,  some  $80,000,000  is  spent  under  contract  research 
programs  in  the  Office  of  Naval  Research.  The  bulk  of  it  is  spent 
in  various  laboratories  and  SYSCOMs,  and  he  is  responsible  for  that 
as  well.  And  we  in  the  Office  of  Naval  Research  are  responsible  for 
that  only  indirectly,  insofar  as  we  might  serve  as  staff  to  him 
regarding  certain  questions  that  he  may  have  and  v;i shes  us  to  be 
concerned  with  in  the  oversight  management  of  that  and  other 
programs . 
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I quite  expected  when  I saw  that  I was  going  to  be  the  last  man  up  that 
I would  be  able  to  say,  "me,  too"  or  "as  he  said"  a lot,  and  am  rather 
surprised  that  there  is  not  more  of  that  in  the  notes  I have  before 
me.  Part  of  that,  I suspect,  is  because  we  approached  the  matter 
differently  than  we  were  told  to  do.  Instead  of  starting  out  with 
the  questions  in  the  box,  I asked  the  group  to  consider  what  one 
actionable  thing  they  would  ask  the  tooth  fairy  to  give  them  to 
improve  the  utilization  of  research  and  development  in  this  people- 
related  area.  1 have  great  confidence  in  the  tooth  fairy;  she  has 
never  let  me  down.  The  group  also  seemed  to  exhibit  a certain  amount 
of  confidence,  so  they  were  willing  to  play  the  game.  We  got  17  or 
18  suggestions,  which  Wally  Sinaiko  and  I tried  to  organize  after  the 
meeting.  What  1 have  before  me  now  represents  an  incomplete  and 
somewhat  fragmented  account  of  what  the  group  had  in  mind. 

The  first  item,  the  need  for  a great  deal  of  cross-education,  cross- 
fertilization, communication,  or  whatever  you  wish  to  call  it,  in 
fact,  was  mentioned  several  times.  It  seems  to  involve  the  idea 
that  it  is  important  for  requirements  to  be  properly  stated  and  reports 
to  go  back  to  the  potential  user  properly  tailored  in  his  language 
and  to  his  needs.  And  this  means  that  reports  may  have  to  deal  with 
specific  cases,  tailored  to  an  individual  user  or  class  of  user.  It 
entails  a great  deal  of  access,  a great  deal  of  opportunity  for  the 
researchers,  research  managers  and  users  to  come  to  know  each  other. 

One  of  the  people  suggested  that  it  has  proved  useful  when  the 
circumstance  exists  for  some  trusted  member  of  the  user  staff  who  is 
well  aware  of  the  concerns  of  his  command  to  sit  in  on  all  of  the 
meetings  that  go  on  and  acquaint  his  command  with  certain  research 
activities  that  may  be  of  benefit,  interest  or  use  in  dealing  with  a 
problem  that  has  just  surfaced  at  a staff  meeting. 

So  there  seems  to  be  a consensus  regarding  the  idea  that  all  members 
need  to  better  understand  how  the  world  works  and  what  the  world 
looks  like.  Someone  summarized  this  by  saying  that  it  would  be 
highly  desirable  for  us  to  develop  a model  so  that  we  would  all 
know  it  is  a complicated  and  complex  world  which  looks  different  from 
the  various  perspectives  of  researcher,  manager,  user,  operator,  high 
seas,  shipyards,  and  so  forth;  and  that  it  would  be  useful  to  have 
a master  model  that  somehow  or  another  puts  this  all  together.  I 
think  we  might,  the  tooth  fairy  might,  be  hard  pressed  to  answer  that 
parti cul ar  wi sh . 

The  second  thing  that  was  discussed  frequently,  avidly  and  noisily 
was  the  role  of  cost-benefit  ana  1 ys i s--who  should  do  it,  when  it 
should  be  done,  what  role  it  should  play,  and  what  should  be  done 
about  research  that  does  not  seem  to  submit  itself  easily  to  that 
kind  of  treatment  (i.e.,  what  do  you  dc  if  you  need  a cost-benefit 
analysis  in  a highly  subjective  area  and  you  can't  produce  one?). 

The  group  agreed  that  cost-benefit  analyses  were  useful,  they  ought 
to  be  done,  they  ought  to  be  done  early,  they  probably  ought  to 
be  reviewed  or  redone  retrospectively,  and  that  there  were  some 
kinds  of  research  for  which  they  were  not  appropriate.  In  these 


F 


cases,  decisions  would  have  to  be  made  as  to  whether  or  not  to  insist 
on  cost-benefit  analysis,  go  along  without  them,  or  refuse  to  do  the 
research  since  cost-benefit  analyses  were  not  available. 

The  role  of  cost-benefit  analysis  in  utilization,  in  case  that  point 
has  escaped  you,  is  that  the  potential  user  can  more  quickly  and 
intelligently  decide  whether  or  not  to  attempt  to  utilize  something 
if  he  has  the  cost-benefit  information. 

Another  point,  that  may  be  related  to  the  first  point,  is  the  apparent 
need  for  formal  mechanisms  that  do  not  exist  in  the  Navy  which  re- 
quire involvement  of  potential  users,  researchers,  and  research 
managers  at  all  stages  of  R6D  from  the  very  earliest  conceptual 
design  or  the  glimmer  in  someone's  eye  all  the  way  through  to  the 
time  that  the  thing  is,  in  fact,  put  in  place  and  someone  can  be 
assured  that  it  is  being  fully  and  effectively  utilized.  When 
being  pressed  for  an  example,  the  individual  who  suggested  this 
indicated  that  he  was  involved  in  software  development,  and  in  that 
area  the  procedures  required  that  you  prepare  functional  specifications 
which  were  discussed  and  argued  out  and  agreed  upon  as  a first  stage. 
And  then  you  go  on  to  the  next  stage  where  the  user  and  the  person 
producing  the  software  were  sitting  on  opposite  sides  of  the  table 
but  were  trying  to  come  to  precise  and  complete  agreement  on  that 
stage,  the  user  recognizing  his  responsibility  for  participating 
avidly  and  intelligently  in  that  process--a  highly  desirable  state 
of  af fa i rs . 

If  I may  be  permitted  a personal  comment,  let  me  say  that  one  of  the 
things  that  troubles  me  is  the  unavailability  of  users  or  potential 
users  who  are  willing  to  join  in  that  kind  of  process  in  such  a 
continuing  and  time-consuming  way.  It  seems  to  me  that  it  would 
be  advisable  for  that  to  happen.  It  has  also  been  my  experience 
that  there  are  many  occasions  where  representatives  of  the  user 
community  plead  that  they  are  too  busy  to  involve  themselves  in  this 
way,  and  it  seems  to  me  that  that  is  very  unfortunate  indeed. 

Back  to  the  group,  in  talking  about  this  interaction  between  re- 
searchers and  users,  one  of  the  members  of  the  group  who  has  had 
many  years  of  experience  as  a researcher  and  has  been  very  effective 
in  producing  research  that  has  gotten  utilized,  pointed  out  that 
we  must  not  make  the  mistake  of  thinking  that  the  user  is  only 
someone  who  lives  in  Washington,  who  reports  to  some  headquarters, 
and  is  a very  senior  person.  Many  users  exist  elsewhere,  at  lower 
levels  of  the  organization.  It  was  his  recommendation  that  re- 
searchers have  an  opportunity  to  involve  themselves  with,  come  to 
know,  and  live  with,  if  necessary,  the  people  at  the  operating  levels 
of  the  organization  in  order  to  know  from  personal  experience  what 
the  problems  are.  Under  those  circumstances,  it  would  be  more  likely 
that  research  would  come  out  in  a useable  form. 

Going  back  to  the  comment  Admiral  Metcalf  made  about  building  bridges, 
one  member  of  the  group  suggested  that  perhaps  we  need  a lot  of 
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different  bridges,  some  of  which  are  just  short  bridges,  such  as  from 
the  basic  researcher  to  the  mission-oriented  researcher. 

Another  theme  that  came  up  and  was  discussed  to  a great  extent,  perhaps 
more  in  this  group  than  in  others,  was  that  research  utilization 
problems  are  by  no  means  limited  to  the  Defense  Department  or  to 
personnel  and  manpower  areas.  They  exist  everywhere.  One  of  the 
members  of  the  faculty  that  was  present  said  he  was  encouraged  by  the 
fact  that  the  Defense  Department  was  willing  to  address  itself  to  the 
problems  of  utilization  and  that  the  researchers  were  so  willing  to 
participate  in  the  discussions,  since  the  agency  that  he  was  affiliated 
with  was  not  yet  willing  to  confront  these  difficult  and  sensitive 
i ssues . 

I would  again  like  to  step  out  of  my  role  of  group  reporter  and  speak 
on  my  own  behalf.  There  was  a lot  of  talk  about  research  management 
and  the  advisability  of  shifting  it  from  one  location  to  another.  I 
made  a comment  to  the  group  about  that  and  would  like  to  repeat  it 
now  because  I think  it  is  very  important. 

Research  management  is  a difficult  "black  art."  It  is  not  something 
that  you  just  assign  to  someone  as  a responsibility  and  hope  that  it 
will  be  done  well.  If  you  do  assign  an  inept  research  manager  to  a 
laboratory,  there  is  no  question  in  my  mind  that  the  quality  of  the 
research  will  plummet.  One  has  to  be  very  careful  in  realigning  the 
research  management  enterprise,  but  that  is  not  to  say  that  it  should 
not  be  realigned.  I am  just  saying  that  you  should  be  certain  that 
the  person  assigned  to  the  job  knows  how  to  do  it  and  has  the  freedom 
and  flexibility  to  permit  him  to  carry  it  out.  Just  putting  that 
title  on  somebody's  door  is  not  going  to  ensure  that  a good  job  will 
be  done. 

Along  these  same  lines,  the  group  pointed  out  that  in  our  business, 
particularly  in  certain  aspects,  there  exists  a kind  of  "catch  22." 

The  user  won't  become  a "true  believer"  unless  he  sees  it  on  his  ship 
and  his  watch,  but  at  the  same  time  he  may  not  allow  the  researcher 
to  collect  any  data  or  do  any  work  on  his  ship  and  watch.  This  is  a 
serious  problem  that  presently  is  handicapping  the  Navy's  research  in 
the  human  resources  area.  We  have  a couple  of  minutes  for  questions. 


DISCUSSION 

QUESTION:  Whose  responsibility  is  it  to  see  that  promising  research 
findings  get  implementation,  and  what  is  the  chief  barrier  to  utili- 
zation? 

DR.  BRYAN:  We  are  all  now  at  the  management  level  and  have  forgotten 
what  it  is  like  to  do  research.  The  scientists  are  interested  in  the 
completion  of  research,  are  anxious  to  move  on,  and  are  not  dedicated 
to  having  the  research  product  implemented;  they  want  somebody  else 
to  do  that.  The  solution  to  this  problem  is  not  to  give  up  implementing 
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the  research  product  but  to  give  the  researcher's  supervisor  the 
responsibility  of  getting  the  research  implemented. 

In  potential  users,  the  major  problem  is  resistance  to  change  and 
the  resistance  is  primarily  because  they  are  quite  busy  doing  other 
things  and  do  not  want  to  take  the  time  that  is  required  to  make 
changes  that  are  necessary  in  order  to  implement  the  research  product. 
So  it  is  primarily  a matter  of  education.  Potential  users  are  not 
sufficiently  educated  that  they  are  the  key  "gate-keepers"  to  research 
utilization.  This  should  be  specified  at  the  very  beginning,  when 
there  is  an  agreement  to  conduct  the  research. 

QUESTION:  Why  don't  researchers  themselves  take  a more  active  job 
in  promoting  utilization? 

DR.  BRYAN:  We  who  have  been  and  are  doing  the  research  generally  are 
not  interested  in  implementation;  we  need  other  groups  of  people 
to  work  with  on  that — a combination  of  officer  personnel  and  civilian 
ope  rators . 

DR.  UHLANER:  I would  like  to  take  o few  moments  to  make  some  comments 
which  I hope  will  give  us  a guide  f some  of  this  week's  discussions. 
As  I was  listening  to  this  report,  i got  the  feeling  that  the  working 
groups  were  excellent  with  respect  to  diagnosis  but,  with  the  possible 
exception  of  some  of  the  remarks  from  Glenn,  I think  much  more  work 
needs  to  be  done  in  terms  of  planning  action  steps. 

A second  thought  I had  is  that  it  is  becoming  very  clear  that  no 
one  utilization  strategy  will  do  the  job.  You  need  a whole  variety 
of  strategies  which  are  appropriate  to  the  different  aspects  of  the 
program.  It  is  very  clear  that  one  that  might  be  used  for,  say 
a 6.3  type  effort,  is  probably  inappropriate  for  6.1  type  effort. 

A third  point  that  seems  to  be  very  critical  here  is  the  fact  that 
in  our  discussions  we've  tended  to  lump  all  the  consumers  together 
as  one  group.  I think  we  have  many  different  groups  to  deal  with. 

It  is  quite  true  that  the  informal  contact  between  the  scientist  in 
the  field  and  the  various  officers  who  have  some  of  the  immediate 
operational  problems  is  one  kind  of  activity.  However,  I do  not 
think  any  of  the  discussions  really  wrestled  with  the  questions  of 
how  a new  large  project  gets  presented  to  the  research  and  development 
advisory  committees  in  each  of  the  services  so  that  such  a project 
can  be  considered  by  the  system  in  competition  with  the  hardware  R&D 
kind  of  project.  It  seems  clear  that  an  entirely  different  set  of 
procedures  are  required  to  achieve  this.  For  one  thing,  those  people 
are  very  much  aware  that  when  they  underwrite  a particular  project 
it  may  be  as  much  as  five  or  six  years  before  the  project  bears  fruit 
for  a particular  service.  That  realization  in  itself  is  a starting 
point  in  terms  of  the  utilization  process. 

Another  thing  that  has  not  been  stated  is  that  all  of  the  utilization 
discussions  are  really  part  of  program  development,  and  unless  one 
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looks  at  the  life-cycle  management  of  the  total  program  development, 
it  may  very  well  be  that  we  will  be  utilizing  our  own  approach  to 
utilization  in  a very  scattered  and  fractionated  way.  You  have  to 
relate  to  the  entire  life-cycle  management  context. 

1 would  like  to  strongly  endorse  the  point  that  was  made  that  we  are 
short  on  large-scale  application  engineering  expertise.  I,  for  one, 
do  not  believe  that  this  is  an  activity  that  can  be  manned  by  junior 
or  lower-level  people.  It  is  a very  challenging  and  high-level 
activity.  What  I do  see  is  a great  deal  of  potential  for  developing 
that  kind  of  application  capability.  We  have  very  high-level  appointed 
officials  in  positions  such  as  assistant  secretaries,  deputies,  and 
so  on,  or  in  the  uniformed  staff  divisions  of  the  various  services, 
but  rarely  do  we  have  an  organization  whose  job  is  application  engi- 
neering on  a high  level  and  with  the  great  deal  of  skill  which  it 
obviously  requires. 

Let  me  see  if  I can  pull  together  some  of  the  things  that  were  empha- 
sized. Almost  all  of  the  presentations  mentioned  the  need  for  Improved 
procedures  of  some  kind  to  facilitate  relations  with  the  consumer. 
However,  I have  not  heard  addressed  the  problem  of  what  happens 
when  a consumer's  requirements  exceed  your  capacity.  Unless  the 
other  services  are  different  from  my  own,  the  Army,  the  consumer's 
requirements  typically  do  exceed  the  available  resources.  In  our 
case,  it  runs  from  3 or  k to  ] . It  is  very  clear  that  the  consumer 
who  brings  in  requirements  thinks  that  his  are  very  important  and, 
indeed,  to  him  they  are.  What  happens  when  you  as  the  research 
manager  or  lab  director  say  to  him,  "Well,  we  are  not  able  to  meet 
your  requirement."?  Is  he  likely  to  come  back  the  following  year 
or  the  year  after  that?  What  I am  leading  up  to  is  that  it  seems 
there  is  a real  need  for  a procedure  for  prioritizing  research 
requirements  which  is  not  done  by  the  laboratory  but  rather  by  a 
community  of  users.  I don't  care  what  the  particular  format  is. 

We  need  to  avoid  getting  into  the  situation  that  because  training 
is  very  powerful,  personnel  and  manpower  are  sold  short.  Some  kind 
of  balance  is  essential  and  I do  not  think  we  have  really  worked  out 
a procedure  where  we  can  guarantee  some  orderly  process  where  this 
balance  is  guaranteed. 

Along  this  same  line,  I think  it  is  very  critical  that  DDRCE  also 
be  apprised  of  what  the  problems  are  in  terms  of  such  issues  as 
what  is  appropriate  by  way  of  successful  products  which  are  worthy  of 
utilization  in  research  categories  6.1,  6.2,  etc.  I have  heard  from 
prior  groups  who  admittedly  are  more  academic-minded  people  that  they 
felt  a 12  percent  success  rate  of  utilization  was  quite  good.  Now 
here  we  are  talking  about  58  percent  being  inadequate.  1 am  not 
proposing  to  answer  it,  but  I do  think  it  is  a rather  important 
question.  Why?  It  seems  to  me  that  we  can  probably  project  a 100 
percent  success  rate  by  structuring  the  problems  in  such  a way  that 
they  are  very  mundane,  very  unimaginative,  and  very  low  risk.  If 
I predicate  that  we  are  going  to  build  a new  form  of  vocabulary  test, 
the  likelihood  of  that  failing  is  almost  zero.  But  if  I predicate 
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that  we  are  going  to  do  some  research  on  the  improvement  of  officer 
performance  for  purposes  of  command,  then  I think  we  have  an 
appreciably  higher  risk  situation. 


The  next  point  that  I think  needs  some  attention  in  the  group  is 
the  question  of  how  one  makes  use  of  specialized  capabilities  in 
accomplishing  the  overall  task  of  program  development  and  utilization. 
It  seems  that  we  are  assigning  each  of  the  responsibilities  to 
everybody.  I think  there  is  a role  that  the  uniformed  R£D  manager, 
the  lab  director,  the  user,  and  each  of  about  10  to  15  others  would 
play  specific  to  his  particular  specialty.  I do  feel  that  there  is 
need  for  spelling  out  who  does  what  which  way. 

I am  just  curious.  How  many  in  this  room  have  been  involved  in 
defending  what  is  called  an  "unfunded  requirement"  for  a new  project? 
I'd  just  like  to  see  a show  of  hands.  A small  number  of  hands  went 
up,  which  is  what  1 expected.  Now,  how  many  of  you  have  had  to 
explain  a particular  research  result  to  a user  at  any  level  for  his 
use?  I would  have  expected  almost  all  hands  to  go  up.  I think  this 
illustrates,  very  crudely  perhaps,  that  different  aspects  of  this 
total  cycle  really  require  different  kinds  of  skills,  different 
kinds  of  positions,  different  kinds  of  talents,  and  yet  they  all 
have  to  be  brought  together  in  order  to  achieve  the  ultimate  objec- 
tive, which  to  me  would  be  to  get  the  best  possible  product  which 
would  satisfy  the  greatest  number  of  decision  makers  (including  both 
the  Congressional  Committees,  the  GAO,  DDR&E  and  all  the  services) 
for  a given  dollar.  When  we  have  achieved  that  kind  of  thing  then 
it  seems  to  me  that  we  have  almost  reached  an  ideal.  With  that 
thought,  I would  like  to  encourage  the  working  groups  to  continue 
deliberation  on  this  problem. 
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REFLECTIONS  FROII  THE  OPERATIONAL  CilWNTTY 


SUMMARY 


In  this  session  of  the  symposium  several  members  of  the  operational 
aommunity  offered  their  perspectives  of  the  central  issues  in  the 
utilization  of  human  resources  RDT&E. 

CDR  Jack  Davis,  Office  of  the  Chief  of  Naval  Technical  Training,  made 
the  following  observations : 

■ The  research  community  is  too  comfortable  with  itself.  The 
researcher  must  assume  the  responsibility  for  making  himself 
known  to  the  operator  and  for  learning  to  understand  the  oper- 
ational environment. 

. The  research  community  must  become  more  responsive  to  the  users  ' 
needs,  and  this  means  that  researchers  must  be  willing  to  work 
on  problems  that  involve  risk  arid  problems  that  may  not  excite 
them.  Operators  find  it  particularly  hard  to  get  the  attention 
of  the  research  community  on  problems  that  require  a short 
response  time. 

. In  the  area  of  people-related  research  the  program  category 
structure  (6.1,  etc.)  is  probably  too  rigid  to  serve  the  needs 
of  the  user  and  may  need  to  be  modified. 

. The  researcher's  obligation  does  not  end  with  the  technical  re- 
port. He  has  not  fulfilled  his  responsibility  until  the  product 
is  utilized. 

LCDR  Lee  Mairs,  Bureau  of  Naval  Personnel,  emphasized  these  points: 

. Cost-benefit  analysis  is  the  key  to  improved  research  utilization. 

The  operator  resists  change  because  he  sees  definite  implementa- 
tion costs  and  indefinite,  unquantified  benefits. 

. The  research  community  must  face  the  fact  that  it  is  now  in  a 
market  environment  and  must  enter  the  marketplace  a>id  demonstrate 
the  value  of  its  product. 

. The  responsiveness  of  the  operator  to  new  R&D  products  is  strongly  \ 

affected  by  the  variable  quality  of  research.  From  the  operator's 
viewpoint,  good  research  implies  both  quality  and  relevance  to  the  I 

operator's  problems.  1 

LCOL  Ernest  Rider,  Air  Force  Manpower  Military  Personnel  Center,  cited 
a number  of  user  concerns: 

. Researchers  tend  to  want  to  "do  their  own  thing, " whereas  the 
"bottom  line"  for  users  is  how  much  the  research  helps  them  in 
their  job. 

. The  timeliness  of  research  often  has  a strong  impact  on  its  poten- 
tial for  utilization.  Too  many  R&D  projects  take  longer  than 
planned  and  thus  lose  their  value.  Better  management  and  better 
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oorrmuniaation  between  researcher  and  user  is  required  to  deal 
with  this  problem. 

• Researchers  don’t  provide  enough  assistance  with  application. 

LCOL  Rider  also  acknowledged  shortcomings  on  the  user  side,  such  as 
lack  of  continuity  (sterming  from  rotation) , inadequate  planning  for 
implementation  funding,  and  failure  to  consider  political  constraints 
on  R&D  utilization. 

LCOL  Frank  Whaley,  USMC,  Naval  Ocean  Systems  Center,  pointed  to  the 
need  for  users  to  do  a better  job  of  specifying  R&D  requirements , and 
for  the  research  community  to  do  a better  job  of  selling  its  capabil- 
ity to  Marine  general  officers. 


REFLECTIONS  FROM  THE  OPERATIONAL  COMMUNITY 


I 


r 


OBSERVATIONS 

DR.  SANDS:  Due  to  a change  that  we  made  in  our  program  yesterday, 
we  will  now  hear  some  "ef lections  from  the  operational  community. 
Without  further  delay,  I would  like  to  introduce  Colonel  Whaley, 
United  States  Marine  Corps,  who  will,  in  turn,  introduce  the  other 
panelists.  Colonel  Whaley... 

COL  WHALEY:  Good  morning,  gentlemen.  Without  further  ado,  let  me 
introduce  Commander  Jack  Davis,  USN,  Chief  of  Naval  Technical 
Training,  as  the  first  speaker  on  our  panel. 


CDR  J.  D.  Davis,  USN 
Assistant  for  Management 
Chief,  Naval  Teahniaat  Training 


I would  like  to  present  perspectives  of  the  R&D  community,  the  R£D 
environment  and  R&D  efforts  from  a fleet  point  of  view  in  general, 
and  from  a shore-establishment  user  point  of  view  in  particular.  I 
think  that  we  view  the  research  community  as  one  that  is  comfortable 
with  itself,  ur.  Rostker  spoke  of  this  last  night,  I believe,  when 
he  said  it  is  comfortable  for  the  research  community  to  work  within 
the  protective  structure  of  the  system  as  it  is.  We  agree  with  that. 

In  the  fleet,  we  do  not  really  know  who  the  researcher  is;  we  do  not 

know  what  he  does.  Every  now  and  then  we  see  some  guy  who  is  not 

one  of  us,  who  looks  different,  who  dresses  differently,  and  who  hands  out 

a lot  of  questionnaires  and  asks  questions.  Generally,  he  does 

know  us  either.  Yesterday  our  subcommittee  made  an  observation  that 

the  researcher  needs  to  learn  to  really  understand  the  environment 

in  order  to  communicate  and  facilitate  the  transfer  of  research.  I 

subscribe  to  that  wholeheartedly.  But  at  the  same  time,  yesterday 

we  heard  several  comments  from  some  of  the  panelists  and  other  members 

about  the  need  to  educate  the  user  to  the  research  problem.  Outside 

of  the  subcommittee,  I never  heard  the  user's  side  presented. 

I would  suggest  that  the  responsibility  is  on  the  researcher's 
shoulders.  We  work  for  the  Navy,  in  particular  for  DoD,  and  the 
object  of  the  game  is  to  support  the  operating  forces  and  enable  the 
commanders  to  do  their  Job  better.  It  is  not  their  responsibility 
to  know  you;  it  is  your  responsibility  to  know  them. 

In  general,  the  research  community  is  not  responsive  to  the  user's 
needs.  When  we  cite  a problem  and  state  our  needs  in  precise  enough 
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terms  that  they  can  be  addressed,  we  are  not  shotgunning  the  effort; 
we  want  that  problem  attacked  even  though  it  may  not  be  especially 
interesting  or  exciting  to  a particular  researcher. 

We,  in  the  training  command,  have  been  particularly  frustrated 
at  our  inability  to  get  help  on  our  short-term  efforts,  something 
that  might  take  only  one,  two,  or  three  months.  The  system  either 
cannot  or  does  not  want  to  respond  to  us  in  that  time  period.  When 
we  cite  a problem  of  this  sort  and  ask  for  help,  generally  we  are 
told  that  it  is  a management  problem,  not  a research  problem.  Well, 
that  is  a myth!  Or  maybe  it  is  a management  problem,  but  the  point 
is  that  the  problem  still  remains.  There  is  no  sensitivity  to 
solving  the  problem. 

If  we  structure  things  rigidly  and  unionize  it  to  the  point  that 
this  is  management  and  this  is  research,  and  we  define  it  very 
precisely,  and  we  cannot  cross  over  it,  that  is  really  no  help  to 
us.  Maybe  it  is  not  a very  clear-cut  research  effort,  but  it  is 
certainly  not  a clear-cut  management  problem.  What  I am  saying 
is,  "Take  the  risk--enter  into  something  that  may  not  be  a precisely- 
defined  research  area." 

Last  night.  Dr.  Rostker  also  talked  about  the  problem  of  rigidity 
of  the  structure  in  terms  of  what  is  6.1,  6.2  up  to  6.5.  He 
thought  this  structure  was  perhaps  appropriate  for  the  hardware 
research  area  but  certainly  not  for  the  personnel  research  area.  I 
agree  with  that;  I guess  the  bottom  line  is  that  we  have  to  change 
the  system.  If  we  have  to  change  the  structure  in  order  to  meet 
some  very  real  needs  of  the  user  community,  then  let  us  do  that. 

Let  us  not  get  to  the  point  of  thinking  that  we  have  fully  discharged 
our  responsibilities  once  the  research  is  done  and  we  have  turned 
out  the  technical  report,  because  I really  do  not  think  that  this  is 
the  case.  If  it  is  good  work  (hopefully,  it  is),  the  responsibility 
really  is  not  discharged  until  it  is  utilized,  until  we  make  the 
user  aware  of  what  we  have  done,  its  value,  its  merit,  its  potential, 
and  press  the  opportunity  to  go  ahead  and  utilize  it.  Thank  you. 


DISCUSSION 

DR.  GLASER:  Does  anyone  have  a question  for  Commander  Davis? 

(JUESTION;  I wonder  if  Commander  Davis  isn't  putting  too  much  of  a 
load  on  R&D  in  asking  them  to  provide  help  in  studying  all  sorts  of 
operational  problems  and  needs. 

CDR  DAVIS:  Well,  I would  be  if  I'd  said  that,  I didn't  mean  that; 

I understand  that  there  are  problems  we  must  solve  ourselves,  and 
we  attempt  to  do  that.  What  I was  saying  is  that  there  are  problems 
that  we  identify  which  in  our  opinion  can  be  solved,  or  at  least 
attacked,  by  the  research  community.  And  then  we  get  into  this  "V/ho 
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shot  John"  kind  of  a thing.  "No,  it's  not  us,  it's  management," 
or  it's  something  else.  The  fact  is  that  there  is  a problem,  and 
we  can't  solve  it.  V/e  ask  the  research  community  for  help,  and  they 
really  don't  think  it's  their  problem.  We're  sort  of  a voice  in 
the  wilderness  looking  for  help.  We're  asking  the  research  community 
to  take  a chance.  Maybe  it  really  isn’t  research  in  the  purest 
sense,  but  the  R&D  community  has  the  ability  to  attack  it  better 
than  we  do.  They  have  people;  they  have  the  money;  they  have  the 
wherewithal  to  get  on  with  it.  We  don't.  That's  what  I was 
suggest i ng . 

QUESTION:  To  what  extent  do  you  believe  the  inability  of  the  re- 
search community  to  be  responsive  is  caused  by  the  failure  of  the 
user  to  make  clear  what  the  real  problem  is  and  what  its  impact 

is.  And  there's  the  problem  of  continuity  of  user  personnel.  What 
exactly  do  you  feel  that  RSD  is  not  responsive  to? 

CDR  DAVIS:  I guess  I don't  fully  understand  the  question.  I guess 
it's  a problem  of  perspective.  If  we  have  been  able  to  identify  a 
problem  and  service  it  to  the  point  that  we  can  even  approach  the 
research  community,  working  through  our  chain  of  command,  then  that, 
in  effect,  should  establish  certain  credibility.  If  that  has  been 
done  it  probably  is  a fairly  significant  problem  and  one  that  needs 
to  be  addressed.  The  problem  is  that  there  are  times  that  we  just 
don't  seem  to  be  able  to  work  the  channel  to  our  advantage  to  get 
answers  to  questions. 

What  we  often  have  to  do  when  we're  tasked  with  a job  is  go  around 

it,  or  under  it,  or  over  it.  I've  secured  research  without  ever 
having  talked  to  any  of  the  people  who  I really  was  supposed  to  be 
talking  to,  but  I was  turned  off  early.  I was  told,  "Gee,  you 

can't  do  it  because  there's  policy,  there's  rules,  there's  regulations, 
there's  instructions."  Much  of  the  time  when  we  really  need  some- 
thing done,  I know  I'm  not  going  to  get  anywhere  by  the  direct 
approach,  so  I say  the  hell  with  it,  and  I go  around  it,  and  I get 
it  done. 

DR.  UHLANER:  I think  this  is  a very  fruitful  kind  of  discussion,  a 
very  good  dialogue,  but  it  is  dealing  entirely  in  generalities.  If 
the  research  community  and  the  management  community  and  the  user 
are  going  to  have  a dialogue,  you  have  to  start  talking  specifics. 

It  would  help  if  you  could  cite  one  or  two  problems,  subjective  as 
they  may  be,  so  that  the  people  here  can  get  some  notion  of  what 
the  frustration  is,  so  that  we  can  at  least  start  thinking  in  terms 
of  whether  there's  a way  around  the  cumbersome  system.  Otherwise 
we're  going  to  go  home  after  each  one  has  made  his  great  flowery 
speech  in  generalities  and  I don't  think  really  anything  will  be 
ga i ned . 

CDR  DAVIS:  All  right,  I realize  that  I am  talking  in  generalities. 

I don't  think  it's  proper  to  name  names  and  point  fingers.  We've 


-63- 


T 


done  this  internally.  What  we're  saying  is  that  we'd  all  agree  that 
there's  a million  children  starving  in  India,  and  some  guy  says,  "Name 
one."  Hell,  I can't  name  one. 

DR.  UHLANER:  That's  not  what  I'm  saying.  You're  dodging  my  question. 
I'm  asking  you  to  mention  a technical  problem,  mention  a need , mention 
a requ i renient  that  in  your  opinion  needs  attention  and  that  is  a 
reasonable  responsibility  for  the  research  community  to  attend  to. 
Whether  we  agree  or  not  is  irrelevant. 

CDR  DAVIS:  An  example  would  be  a good  short  look  at  why  attrition 
rates  at  Recruit  Training  Command  San  Diego  doubled  in  a one  month 
period  of  time. 

DR.  UHLANER;  Beautiful.  Do  you  feel  that  you  can  get  an  answer  to 
that  problem  in  three  months? 

CDR  DAVIS:  I would  hope  so. 

DR.  UHLANER:  Now  that's  the  kind  of  dialogue  we  should  be  having. 

QUESTION;  Other  commands  have  recognized  the  problem  you  described 
and  have  developed  a mechanism  for  addressing  it.  During  the  Viet  Nam 
war  the  Navy  adopted  a system  used  by  the  Army,  calling  it  by  a 
different  name,  the  Science  Assistance  Program.  This  program  has 
science  advisors  located  at  various  Navy  commands  around  the  world. 

The  science  advisor  has  the  ability  to  describe  a problem  and  send  it 
back  to  the  Director  of  the  Navy  Science  Assistance  Program,  who  has 
the  funds  to  task  the  appropriate  research  organization  to  address 
the  problem.  I would  be  happy  to  describe  the  program  to  you. 

LCOL  WHALEY:  Thank  you.  I think  we'd  better  go  on  or  we'll  end  up 
running  the  ship  aground.  Lieutenant  Commander  Lee  Hairs  is  in  the 
economic  analysis  section  of  the  Chief  of  Naval  personnel.  He'll 
give  you  the  viewpoint  of  the  Navy  side  of  the  house. 


LCDR  Lee  Hairs,  USN 
Head,  Economic  <§  Cost  Analysis  Section 
Bureau  of  Naval  Personnel 


In  our  discussions  at  breakfast  this  morning,  when  we  were  talking 
about  what  we  were  all  going  to  say  here,  we  felt  that  Dr.  Rostker 
stole  a lot  of  our  thunder  last  night,  and  if  that  sounds  like  an 
indictment  (of  the  R&D  community)  believe  me,  it  is.  During  the  small 
group  that  I was  in  yesterday,  there  was  a tremendous  discussion  as 
reported  in  the  plenary  session  about  cost-benefit  analysis--about 
whether  it  should  be  done  or  shouldn't  be  done,  etc.  The  pro  side 
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turned  out  to  be  the  economists  in  the  room  and  the  con  side  was  the  ' 

other  disciplines.  Let  me  tell  you  why  I think  cost-benefit  analysis 
is  so  important  and  how  it  perhaps  might  be  the  key  in  unlocking  the 
door  to  better  research  utilization.  Admiral  Metcalf  spoke  yesterday 
about  the  resistance  to  change  that  develops  when  it  comes  to  im- 
plementing any  sort  of  RSD  product.  I would  like  to  submit  to  you 
that  when  you  look  at  it  from  the  operator's  point  of  view,  the  • 

resistance  to  change  stems  from  the  lack  of  any  quantified  benefits 
in  doing  the  change  in  the  face  of  very  definite  and  certain  imple- 
mentation costs.  And  if  the  operator  is  not  given  this  kind  of  data, 
he  feels  that  he  just  cannot  take  his  time  to  do  this  work  himself. 

I think  this  is  the  key  that  tne  research  and  development  community 

is  going  to  have  to  recognize,  and  start  to  use  it  to  unlock  the  door.  I 

In  the  private  sector,  look  at  Xerox,  Kodak,  and  IBM.  They  do  not  sit 
around  waiting  for  the  customers  to  come  to  them.  They  have  salesmen 
that  sally  forth  into  the  marketplace  giving  out  information  and 
seeking  out  new  customers.  We  do  not  see  this  in  the  military  research 
and  development  community.  You  gentlemen  are  now  operating  in  a 
market  environment.  In  that  environment  R&D  funds  can  be  shifted 
by  the  program  managers  to  buy  chipping  hammers,  paint,  and  tooth- 
picks; so  if  they  do  not  see  that  there  is  going  to  be  some  value 
from  research  and  development,  believe  me,  the  pot  is  going  to 
diminish. 

During  our  group  discussion  yesterday  I heard  a lot  of  complaint  that 

some  research  areas  are  not  amenable  to  cost-benefit  analysis.  I 

would  like  to  submit  to  you  that  if  there  are  a certain  subset  of 

cases  where  this  is  true,  it  certainly  does  not  preclude  all  other 

pieces  of  research  from  cost-benefit  analysis  techniques.  Furthermore, 

even  in  so-called  "soft"  areas  there  are  "hard"  aspects.  The  benefit 

of  race-relations  programs  was  one  area  that  was  thrown  up  to  me 

yesterday  as  defying  cost-benefit  analysis.  I submit  that  that  can 

be  quantified,  that  the  dollar  value  of  resources  that  the  Navy  now 

spends  on  race-related  discipline  problems  can  be  estimated,  and 

the  amount  of  money  that  is  spent  on  race- re  1 at  ions  programs  now 

could  be  a savings  if  somebody  in  the  R&D  community  could  develop 

a pill  or  something  from  the  tooth  fairy  that  could  eliminate  that 

problem  overnight.  I would  like  to  refer  some  of  you  people  to  a 

book  by  Arnold  Harberger  at  the  University  of  Chicago  entitled  Project 

Eva  1 uat ion , in  which  he  talks  about  the  techniques  of  cost-benefit 

analysis  in  these  fuzzy  areas.  For  example,  the  value  of  human  life 

has  been  determined  for  cost  benefit  analysis  purposes.  Now  of 

course,  if  it's  my  life  I tend  to  value  it  a lot  higher  than  perhaps 

anyone  else  will  in  this  room,  especially  after  I have  been  talking 

for  a little  while.  But  when  you  talk  about  life  in  the  abstract  and 

perhaps  have  a program  that  increases  safety  or  reduces  the  probability  i 

of  death,  then  there  are  market  estimates  of  how  much  money  should  be 

spent  for  this.  It  is  not  infinite;  if  it  were  infinite  we  would  ban 

cars.  But  these  are  the  techniques  that  the  R&D  community  are  going 

to  have  to  sharpen  and  really  take  a strono  look  at. 

I 

I 
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ResistoDce  to  change,  I would  underline  again,  stems  from  a lack  of 
quantified  benefits  in  the  face  of  absolutely  certain  costs  of 
trying  to  implement  any  change.  This  is  what's  got  to  be  changed. 

A minor  point  I would  like  to  bring  up  is  that  the  quality  of 
research  has  a lot  of  variability.  And  the  cost  that  the  R&D 
community  suffers  from  a piece  of  bad  research  is  not  balanced  out 
by  two,  three  or  four  pieces  of  good  research.  All  it  takes  is  for 
the  operator  sitting  in  Washington  to  read  a paper,  supposedly  using 
discriminant  analysis,  that  tells  him  that  reenlistment  bonuses  have 
no  effect  on  retention  because  this  researcher  notes  that  only  those 
ratings  that  have  low  retentions  are  receiving  the  high  reenlistment 
bonus  payments.  Now  that  is  just  absolutely  poor  research.  I have 
a long  list  of  examples  of  bad  research  that  I left  at  home.  But 
there  good  research,  and  this  is  the  research  that  identifies 
the  problems,  that  goes  after  the  relevance  that  Dr.  Rostker  talked 
about  last  night.  For  example:  the  work  that  NPRD^  has  done  with 
all  of  ADSTAP  models  is  of  immediate  relevance  to  the  Chief  of 
Naval  Personnel  and  was  virtually  put  on  line  and  used  before  the 
technical  reports  were  even  written.  Other  examples  of  good 
research  are  the  work  of  Stan  Horowitz  at  CNA  in  quantifying  the 
benefits  of  personnel  manning  relative  to  the  mean  time  to  repair 
between  breakdown,  and  the  work  of  Bob  Lockman  at  CNA  in  developing 
pre-service  variables  to  use  for  screening  enlistments,  resulting  in 
reduced  attrition.  All  of  this  was  in  response  to  urgent  problems 
within  the  operational  community  and  virtually  before  the  reports 
were  written  the  research  was  already  implemented.  These  kinds  of 
good  research  are  things  that  other  researchers  should  look  at  to 
see  how  they  can  model  their  work  after  it.  Any  questions? 

QUESTION:  I have  a comment  I would  like  to  make.  I'm  disturbed 

by  the  factual  information  and  the  way  it  is  selectively  perceived. 

I was  in  that  session  yesterday  and  I didn't  perceive  the  clear  cut 
distinction  between  the  economists  and  the  other  disciplines.  In 
this  conference  there  has  been  too  much  "them  and  us",  too  much 
finger  pointing.  As  a matter  of  fact,  the  Defense  Science  Board 
made  a special  point  that  the  RSD  community  is  making  serious 
attempts  to  include  cost-benefit  analysis  in  the  area  of  training. 

LCOL  WHALEY:  Lieutenant  Colonel  Ernie  Rider  is  Chief  of  the  research 
team  at  the  Air  Force  Military  Manpower/Personnel  Center,  and  he 
will  give  you  the  Air  Force  picture. 


LCOL  Ernest  Rider,  USAF 
Chief,  Human  Resources  Research  Team 
Air  Force  Military  Manpower/Personnel  Center 
Randolph  Air  Force  Base 


I probably  should  preface  my  remarks  this  morning  by  saying  that  I 
represent  the  Air  Force  users  to  some  extent  since  I see  all  of  the 
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research  requests  that  come  from  agencies  external  to  the  Air  Force 
Systems  Command  that  are  typically  levied  upon  the  Air  Force  Human 
Resources  Laboratory.  This  covers  a wide  variety  of  users  anywhere 
from  the  trainers  in  your  Training  Command  to  the  operators  in  the 
other  operational  commands  and  the  Air  Staff  agencies  both  in  the 
personnel  and  manpower  areas.  We  typically  coordinate  these  requests 
with  the  Air  Staff  and  .make  sure  there  is  higher  level  recognition  of 
the  problem.  It's  a little  hard  to  address  the  interests  and  concerns 
of  all  of  these  users  in  a very  short  time,  but  I might  make  a 
few  comments  about  my  overall  impression  in  the  few  short  years  in 
which  I have  looked  at  this  business.  I'm  sort  of  objective  in  my 
statements  here  because  I have  never  been  a researcher  per  se  in  the 
human  resource  area  and  I represent  all  types  of  users  within  the  Air 
Force.  I've  been  involved  for  about  three  years,  so  I guess  I have 
a pretty  good  sample  of  what's  gone  through  the  system  and  what 
people's  reactions  have  been. 

The  bottom  line,  I believe,  on  how  people  use  the  Human  Resources 
Laboratory  is  how  good  a job  it  does  for  them,  how  much  it  helps 
them  in  their  particular  jobs,  how  much  it  helps  them  in  their 
particular  policy  decisions.  Obviously,  don't  expect  all  of  the 
research  to  work  out  well.  My  impression  is  that  roughly  half  of 
the  "research"  projects,  i.e.,  the  ones  that  really  require  experi- 
mentation and  considerable  time,  probably  work  out  pretty  well  and 
people  think  well  of  the  research.  I think  that's  probably  about 
the  best  you  could  hope  for  in  the  human  resources  RED  area. 

The  laboratory  also  provides  some  valuable  help  on  a short-term 
basis  through  what  we  call  consultative  assistance.  I think  they've 
got  a wide  base  of  knowledge  and  a lot  of  experience  which  is  very 
valuable  to  the  operators.  This  is  especially  clear  in  the  flight 
simulation  area,  in  which  they  have  done  an  awful  lot  of  good  work. 
People  are  depending  on  them  for  good  advice  as  to  what's  appropriate 
to  do  in  these  areas  as  far  as  purchase  and  development  of  simulators. 
They  also  have  a very  valuable  historical  data  base,  that  is  com- 
puterized. It  contains  personnel  and  training  records,  which  are 
very  valuable  for  short-term  analysis  in  addressing  policy  questions. 

Now  that  I've  said  good  things  about  the  laboratory  I'm  going  to  make 
a few  comments  about  areas  where  improvement  could  come  about  and  I 
would  guess  that  the  Navy  and  Marine  representatives  who  we're  about 
to  hear  from  may  have  similar  things  to  say.  I think  that  basically 
the  researchers  want  to  do  their  own  thing,  and  that  may  be  necessary, 
but  to  the  operator  it  appears  that  they  don't  always  get  as  close  to 
the  project  as  would  be  desirable.  From  the  operator's  viewpoint, 
the  quality  of  the  research--the  applicability  and  understandabi 1 i ty 
of  the  results  as  far  as  they  relate  to  the  operational  problems-- 
could  be  improved. 

The  time  dimension  in  which  research  is  conducted  is  another  concern 
to  people.  The  operator  who  has  a problem  would  like  a solution  in 
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the  most  expeditious  manner.  He  usually  recognizes  that  it  will  take 
some  time  to  do  good  research.  But  frequently  (for  many  reasons, 
some  of  which  are  operator  problems  and  some  researcher  problems)  the 
research  takes  longer  than  originally  planned,  falls  behind  schedule, 
and  may  not  be  of  as  much  use  as  if  it  had  been  done  on  schedule.  I 
guess  i t al 1 boils  down  to  a need  for  better  management,  better 
communication  between  user  and  researcher,  to  help  keep  things  on 
track.  Another  problem  area  involves  the  application  phase.  The 
Human  Resources  Laboratory  is  now  cranking  up  to  give  more  assistance 
in  this  area.  They  are  going  to  be  doing  the  cost-benefit  analysis 
in  selected  areas.  Someone  commented  earlier  that  some  areas  are  less 
amenable  than  others  to  this  type  of  analysis.  I think  cost-benefit 
analysis  should  certainly  be  done  for  the  major  projects.  And 
since  the  operators  typically  do  not  understand  the  research  well 
enough,  ( think  the  researchers  should  do  the  analysis. 

Though  they  do  a very  good  job  in  most  areas,  one  area  where  re- 
searchers could  improve  is  in  putting  their  ideas  on  paper  in  a 
timely  manner,  and  in  a manner  that  means  something  to  the  operators 
and  to  the  operational  community.  I don't  really  think  they  communi- 
cate well  enough  with  the  users,  particularly  in  the  application 
phase.  There  is  a salesmanship  aspect  of  research  which  I think 
several  people  have  alluded  to.  If  you  are  going  to  improve  your 
batting  average  on  application,  that  is  a factor  which  needs  to  be 
considered.  I guess  all  of  these  things  will  sort  of  "come  out  in 
the  wash"  if  there  is  increased  communication  between  the  users  and 
the  research  managers.  Hopefully,  the  communications  process  can  be 
formalized  a little  bit  more.  I think  in  the  Air  Force  we  have  a 
leg  up  on  the  formalization  of  the  procedures.  We  have  an  Air 
Force  regulation  which  I think  Dr.  Clark  mentioned  yesterday.  We're 
thinking  about  increasing  the  bureaucracy  to  some  extent  to  force 
people  to  communicate  during  a research  project  where  it  is  appropriate, 
and  hopefully  we  will  be  hearing  a lot  more  good  things  from  the  users. 
We  hear  a lot  of  good  things  now,  but  we  also  hear  a lot  of  criticisms. 

I would  like  to  summarize  my  comments  briefly  by  saying  that  there  is 
concern  by  users  about  the  quality  of  research,  about  its  timeliness, 
and  about  the  fact  that  researchers  generally  don't  provide  enough 
assistance  with  application.  Some  users  do  not  always  do  their  job 
either.  There  is  not  always  continuity  of  the  people  who  are 
sponsoring  the  research.  As  you  know,  we  move  frequently  in  the 
military.  We  need  to  make  sure  on  a more  systematic  basis  that  when 
people  do  move  we  get  replacements  who  continue  to  work  with  a given 
project.  And  I guess  users  don't  always  adequately  consider  the 
application  phase  of  a project  either.  They  don't  always  do  a good 
job  of  planning  for  implementation  funds  and  they  often  fail  to 
consider  the  political  constraints  which  may  be  operating  depending 
on  the  results  and  types  of  research.  Overall,  I think  RsD  has 
provided  an  awful  lot  of  important  inputs  to  the  personnel  and  train- 
ing areas.  I think  it  could  be  improved.  The  question  is,  how 
much  can  we  improve  it  without  making  researchers  spend  so  much  time 
on  utilization  that  there  is  not  enough  time  available  for  research? 
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Let  me  start  off  by  saying  that  the  Marine  Corps  organization  for 
personnel  research  and  development  is  in  disarray.  Let  me  add  that 
on  the  user’s  side  of  the  house  we  have  not,  heretofore,  really 
gotten  our  act  together.  A couple  of  us  are  working  together  to  get 
our  requirements  clearly  stated  so  that  we  understand  them.  And 
then,  hopefully,  we  will  transmit  this  to  the  RSD  people.  So,  I do 
realize  that  our  personnel  R&D  activity  is  in  trouble  in  the  sense 
that  at  the  present  time  the  user  is  not  laying  requirements  onto 
the  system.  Now,  we  all  understand  that  people  problems  are  para- 
mount with  the  Marine  Corps  because,  and  I am  very  serious  about 
this,  that  is  our  primary  business.  The  Navy  has  ships  and  planes. 

We  have  planes  too,  but  people  in  the  squad,  fire  team,  platoon, 
company,  that's  our  bread  and  butter,  that's  how  we  make  it.  And 
unfortunately,  the  perception  that  exists  among  Marine  Generals 
that  I have  talked  to,  basically  summed  up  is,  "Psychologists  suck." 
And  you  know,  it's  kind  of  difficult  when  I get  back  to  Washington 
to  say,  "We  saw  this  neat  thing  out  at  NPP^DC."  I get  a "You've 
got  to  be  kidding  me."  Of  course  it's  stated  a little  differently 
than  that.  The  R&D  community,  I am  convinced,  is  providing  us  with 
a better  way  to  do  things  because  I have  seen  in  21  years  that  running 
things  based  on  a gut  level  perception  isn't  the  way  to  do  it.  On 
the  other  side  of  the  coin,  we  have  not  utilized  the  system  to 
the  degree  that  we  should,  and  I intend  to  push  the  system  so  that 
we  can.  You  ask  about  a specific.  I have  a great  deal  of  ex- 
perience in  R&D  units  and  I've  been  on  the  other  side  of  the  house 
where  I've  finished  five  years  of  command  time  in  three  different 
organizations,  so  I'm  keenly  aware  of  the  problems  that  we  have  had 
in  our  people  side  of  the  house.  The  specific  is  as  follows:  We 
should  have  a better  way  to  keep  our  troops  soldiering  in  peace  time, 
which  of  and  by  itself  isn't  much  fun.  We  don't  necessarily  have 
to  kill  people  during  peacetime  to  keep  them  gainfully  employed, 
but  we  need  to  have  training  set  up  and  organized  and  run  so  that 
our  people  are  doing  what  they  have  been  trained  to  do,  in  a pro- 
gressive type  manner.  Our  people  who  fix  radios  and  radars  get  a 
chance  every  day  to  do  what  they  came  in  for,  but  our  infantrymen, 
the  main  part  of  the  corps,  don't.  Unfortunately , we  have  a 
shortage  of  ranges,  we  have  a shortage  of  ships,  we  have  a shortage 
of  all  those  other  neat  things  that  we  need  to  train  these  infantry 
guys.  And  you  know,  about  the  73rd  time  you  take  the  same  hill 
you  kind  of  lose  a little  of  the  dynamics  of  the  situation.  So, 
what  we  need  is  something  to  keep  these  people  gainfully  employed  so 
that  they  are  really  getting  something  out  of  soldiering  in  a peace- 
time environment.  The  way  it  is  now,  the  infantry  guy  ends  up 


-69- 


r — 1 


running  our  bases.  You  know,  five  guys  over  in  special  services, 
two  guys  life-guarding,  doing  everything  but  soldiering.  1 don't 
have  the  answer  to  that.  I'm  sure  that  none  of  the  Marine  Generals 
have  the  answer  either.  It's  one  of  those  things  that  you  just 

can't  pull  out  of  your  gut  and  find  a viable  solution.  But  those 
are  the  kinds  of  things  that  1 think  we  need  to  have  addressed. 

And  hopefully  we  will  clean  up  our  act  in  Quantico,  our  RSD  require- 
ments place,  so  that  we  send  something  from  that  level  that  coordinates 
the  hardware  side  with  the  people  side,  because  we  have  kind  of 
gotten  our  hardware  act  together.  Unfortunately  the  paradox  is 
that  our  most  important  problem  is  probably  the  least  addressed. 

So  1 say  to  you,  "Yes,  we  have  some  requirements  homework  to  do,"  and 
this  is  key,  1 think.  The  folks  in  your  community  have  failed  to  sell 
the  pragmatic  approach  to  our  Marine  General  officers,  and  the  junior 
officers  too;  let's  face  it,  the  Majors  and  the  Lieutenant  Colonels 
are  the  ones  who  end  up  getting  in  there  and  fooling  around,  maybe 
getting  something  done  and  maybe  not.  This  hasn't  been  done,  and 
I charge  all  of  you  that  if  we're  going  to  address  this  most  important 
Marine  problem,  you  have  to  get  out  there  and  sell  the  capability 
that  1 honestly  believe  you  have.  But  1 also  am  well  aware  that, 
for  some  reasons  that  I'm  really  not  aware  of,  it  has  not  been 
transmitted  to  our  folks.  That's  kind  of  general,  but  after  all 
my  experience  in  the  business,  that's  where  1 come  off  at. 
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THE  STATE-OF-THE-ART  IN  RESEARCH  UTILIZATION  WITHIN 
OTHER  FEDERAL  AGENCIES 


SUMIARY 


In  this  session  of  the  symposium,  representatives  of  several  non- 
military federal  agencies  reported  their  experiences  in  dealing  with 
the  problems  of  research  utilization. 

Dr.  Howard  Davis,  National  Institute  of  Mental  Health,  undertook  to 
explode  the  "myth"  that  the  key  to  utilization  is  good  research.  He 
cited  a number  of  illustrations  that  good  research  is  not  enough — 
that  simply  being  presented  with  a better  way  of  doing  something 
provides  no  assurance  that  it  will  be  done.  Above  all  it  is  the  hu- 
man factor,  he  argued,  that  determines  whether  knowledge  will  be  ef- 
fectively transferred.  He  pointed  out  that  utilization  is  a matter 
of  planned  organizational  change,  and  that  techniques  exist  for  facili- 
tating change,  such  as  his  own  A VICTORY  model.  A VICTORY  is  designed 
to  assist  in  assessing  the  readiness  of  an  organization  to  undertake 
a given  change — based  on  such  factors  as  its  resources , values,  in- 
formation, circumstances,  motivation,  resistance,  and  perceived  pay- 
off— and  to  assist  in  remediating  defiaiences  in  these  areas. 

Judah  Drob,  Department  of  Labor,  described  the  systematic  approach 
to  utilization  of  R&D  that  is  taken  by  the  Employment  and  Training 
Administration  of  DOL.  This  approach  includes  the  following  elements: 

. Requirement  that  every  R&D  proposal  discuss  utilization,  including 
what  products  the  project  will  yield- 

. Strenuous  effort  to  involve  potential  users,  via  Advisory  Com- 
mittees, site  visits,  briefings,  etc. 

. Use  of  a "buddy  system, " whereby  a member  of  the  DOL  utilization 
staff  has  responsibility  for  representing  the  utilization  con- 
cerns and  plans  of  an  R&D  project  . 

. Recognition  that  training  and  technical  assistance  constitute 
major  steps  in  developing  the  capacity  of  the  user  to  make  use 
of  R&D  products  . 

. Acceptance  of  the  principle  that  "user  orientation"  is  the  first 
commandment,  and  that  it  need  not  compromise  your  rights  and 
integrity. 

Dr.  Meredith  Crawford,  George  Washington  University , presented  an 
historical  view  of  HumRRO’s  struggle  with  utilization.  He  empha- 
sized HumRRO’s  interest  in  the  military  problem  and  commitment  to 
providing  useful  results.  He  cited  several  other  factors  (each 
of  which  may  be  found  in  the  literature  on  knowledge  utilization) 
that  contributed  to  HumRRO's  track  record  in  utilization,  including: 

. A dispersed  organizational  structure  that  facilitated  close 
relations  with  customers  . 
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. Systematic  procedure  for  eliciting  customer  requirements  and 
an  active  effort  to  conduct  a dialogue  to  ensure  a cornnon  under- 
standing of  those  requirements. 

. Use  of  the  experimental  method,  insofar  as  possible. 

. A flexible  work  program,  adaptable  to  the  short-term  as  well  as 
long-term  needs  of  the  customer. 

. Careful  attention  to  presentation  of  results,  emphasizing  quality 
reports,  user  participation  in  briefings,  and  pidblication  of  util- 
ization successes  as  a mechanism  for  diffusion  and  often  multiple 
utilization  of  a product. 

Bert  Griffis,  Rehabilitation  Services  Administration  (ESA),  entertained 
and  stimulated  the  conferees  with  an  eloquent  reminder  that  RSD  is  not 
the  only  source  of  knowledge  that  may  be  utilized  in  the  solution  of 
people-related  problems,  but  rather  that  the  humanities , history, 
philosophy,  literature,  and  the  fine  arts  may  have  abundant  wisdom  to 
offer  as  well.  He  then  described  the  major  elements  of  RSA's  utilization 
program,  including: 

. An  effort  to  emphasize  utilization  in  the  planning  and  conduct  of 
research  (e.g.,  by  making  explicit  and  disseminating  the  utilization 
elements  that  an  R&D  proposal  should  contain). 

. A strong  focus  on  dissemination  of  usable  results  (e.g.,  by  prepara- 
tion and  dissemination  of  abstracts , research  briefs,  and  a guide 
for  preparing  final  reports  to  facilitate  use  of  results). 

. A program  to  establish  human  and  organizational  links  in  the  field 
between  research  and  practice  (e.g.,  funding  regional  research 
utilization  laboratories,  and  conducting  a national  dissemination 
program  of  RU  specialists  in  state  rehabilitation  service  agencies) . 

. A program  of  research  on  research  utilization  (including  evaluation 
of  a large  sample  of  R&D  reports  to  derive  principles  of  effective 
utilization) . 

. An  effort  to  nurture  the  capacity  for  self -initiated  change  in 
service  agencies  (e.g.,  by  stressing  R&D  utilization  in  training 
programs) . 

Finally,  Mr.  Griffis  admonished  the  conferees  to  be  conscious  of  certain 
problems  and  limitations  of  research  utilization,  such  as  the  diffi- 
culty in  preventing  RU  from  becoming  an  end  in  itself,  and  the  tendency 
of  RU  to  contribute  to  the  information  explosion,  rather  than 
to  respond  constructively  to  it. 

Dr.  Ronald  Havelock,  University  of  Michigan,  discussed  efforts  to  in- 
stitutionalize research  utilization  in  the  field  of  education,  with 
special  reference  to  the  activities  of  the  National  Institute  of 
Education  in  this  area,  including: 

. Establishment  of  the  Educational  Resources  Information  Center 
(ERIC)  as  the  first  systematic  step  towards  utilization  of 
educational  research. 

. Transformation  activities , such  as  summarizing,  abstracting  and 
translating  from  scientific  language  to  practitioner  language. 
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. The  use  of  linking  agents  in  several  state  department  of 
education. 

. The  effort  to  establish  a nationwide  system  of  regional  resource 
centers  as  an  institutional  mechanism  for  improving  utilization. 

Dr.  Havelock  suggested  that  the  function  of  research  utilization  be 
regarded  as  a full-time  fob  rather  than  extra  duty,  and  that  it 
requires  funds  and  trained  people.  In  conclusion,  he  referred  to 
the  growing  literature  reporting  research  on  research  utilization. 
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THE  STATE-OF-THE-ART  If'  RESEARCH 

WITHIN  OTHER  FEDERAL  AGENCIES 


DR.  SANDS:  Dr.  Glaser  will  be  chairing  this  session,  so  without  further 
delay,  let  me  give  you  Dr.  Ed  Glaser. 

DR.  GLASER:  Thank  you.  About  195^,  Dr.  Carl  Rogers,  a very  eminent 
psychologist,  and  Dr.  Fritz  Roeth 1 i sberger , a professor  of  management 
of  the  Harvard  Business  School,  wrote  an  article  published  in  the 
Harvard  Business  Review,  entitled,  "Barriers  and  Gateways  to 
Communication."  I think  it  might  be  worth  repeating  the  essence  of 
their  message  because  I think  it  has  some  bearing  on  the  need  for 
better  understanding  between  researchers  and  users.  The  message  was: 
when  you  are  experiencing  a difference  of  opinion  with  someone, 
pause  and  say  something  like,  "Let  me  see  if  I understand  what  you're 
saying,"  and  summarize  back  to  that  person  what  you  think  he  said. 

The  person  then  either  agrees  with  you  or  not;  stay  with  it  until 
the  person  feels  understood.  Then  take  off  and  do  anything  you 
want  in  terms  of  argument,  but  at  least  you  start  from  a base  of 
caring  to  know  accurately  what  the  other  person  has  said.  In  a way, 

Jay  Uhlaner  did  that  this  morning  by  indicating  his  need  for  examples 
so  that  he  could  understand  Commander  Davis'  message.  He  clearly 
indicated  a desire  to  really  tune  in.  That  kind  of  effort,  I think, 
can  really  pay  dividends.  The  people  who  now  are  going  to  present 
their  respective  experiences  with  efforts  to  achieve  research  utili- 
zation in  their  respective  Federal  agencies,  probably  have  been 
wrestling  with  this  problem  of  how  to  facilitate  research  utilization 
longer  than  anyone  else  in  the  United  States.  Our  first  speaker 
is  Dr.  Howard  Davis,  chief  of  the  Mental  Health  Services  Development 
Branch,  National  Institute  of  Mental  Health.  Howie  says  his  real 
claim  to  fame  stems  from  his  being  an  old  Air  Force  bomber  pilot. 


NATIONAL  INSTITUTE  OF  MENTAL  HEALTH  (NIMH) 


Dr.  Howard  Davis 

Chief,  Mental  Health  Services  Development  Branch 
National  Institute  of  Mental  Health 


It's  been  a tremendously  impressive  meeting  so  far  for  one  who  comes 
into  your  group  from  outside.  Beginning  with  Captain  Clarkin's 
comments  yesterday  morning  clear  through  to  the  panel  of  the 
operational  arms  this  morning,  I have  been  most  impressed  by  the 
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sophistication  that  is  held  by  this  group  in  the  area  of  research 
utilization  and  the  problems  that  are  contained  in  it. 

One  of  the  issues  that  I heard  last  night  from  Secretary  Rostker 
and  also  during  the  small  group  discussion,  was  that  the  problem 
may  lie  with  the  researchers.  That  is,  if  you  only  did  good  research 
then,  of  course,  it  would  be  utilized.  I have  heard  this  for  so 
many  years,  particularly  from  our  academic  brethren  in  health 
research.  They  say,  as  someone  did  during  the  small  group  yester- 
day, "Please  tell  us  of  some  good  research  that  hasn't  been  adequately 
used." 

This  matter  came  out  last  week  at  our  meeting  of  federal  research 
managers  at  American  University,  where  again  people  were  challenging 
us  to  give  some  illustrations  of  research  projects  that  were  really 
worth  using  that  hadn't  been  used.  And  I had  to  admit  to  the  heads 
of  the  Department  of  Transportation's  R£D  program  who  were  there,  that 
if  you  people  came  up  with  a mixture  for  pavement  that  would  resist 
freezing  except  at  very  low  temperatures,  then  of  course  the  Northern 
states  would  rush  to  use  it.  I would  have  to  agree  with  Dr.  Glenn 
Bryan  that  if  the  Navy  had  invented  a device  that  would  successfully 
lift  bombs  to  the  wings  of  air  craft  and  effectively  replace  the 
hernia  bars,  then  of  course  the  commands  would  use  it. 

I had  to  agree  at  that  meeting  with  Lou  Caresse,  who  heads  RSD 
policy  for  the  National  Cancer  Institute,  that  if  their  researchers 
ever  came  up  with  some  findings  that  would  really  tell  us  how  we 
could  lower  the  likelihood  of  lung  cancer,  of  course  we  would  use 
it.  My  point  is  that  the  best  disseminated  research  information  is 
on  the  back  of  every  pack  of  cigarettes,  yet  some  of  us  still  keep 
puffing  away  and  the  sales  of  cigarettes  continue  to  go  up  about 
one  percent  a year.  Surely  there  is  something  wrong  in  the  assumption 
that  if  you  just  present  good  information,  change  will  occur.  We 
did  have  a little  humorous  interchange  at  that  federal  research 
managers  meeting,  recalling  that  some  time  ago  the  American  Medical 
Association  had  issued  a series  of  news  releases  on  the  finding  that 
men  who  smoke  run  a greater  risk  of  impotency.  We  played  around 
with  how  much  better  the  research  findings  could  be  disseminated  if 
on  the  side  of  every  pack  it  were  to  say,  "Caution:  the  Surgeon 
General  has  found  that  if  you  smoke  you  may  find  your  craft  ebbing." 

On  that  I think  I should  stop.  That  will  be  my  one  single  message 
this  morning. 

There's  been  so  much  wisdom  expressed  here  that  I am  frankly  a little 
intimidated  about  giving  you  a litany  of  some  of  the  practices  and 
techniques  we  feel  are  advisable  for  increasing  the  likelihood  that 
research  will  be  effectively  transferred.  But  there  is  one  con- 
clusion that  I think  is  incontrovertible,  and  that  is  that  research 
utilization  is  above  all  a matter  of  human  factors.  It's  what 
people  are  i ns i de-- the i r need  systems,  how  they  feel  about  things, 
how  it  comes  across  to  them--that  determines  whether  knowledge 
will  be  effectively  transferred.  I think  I can  actually  say  with 
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absolute  sincerity  that  the  real  way  to  plan  a research  utilization 
program  and  a research  management  model  is  just  one  of  good  common 
sense,  which  I heard  an  abundance  of  yesterday.  Since  I carry  no 
credentials  in  common  sense.  I'll  have  to  fall  back  on  what  the 
psychologists  say.  This  doesn't  mean  that  one  has  to  be  a psycholo- 
gist to  effectively  plan  a good  research  and  utilization  program. 

In  fact,  I think  that  psychologists  usually  muck  things  up  by  getting 
involved  with  all  of  their  theories  and  principles,  and  so  I would 
have  to  ask  to  be  forgiven  for  that,  but  what  else  do  I know  to  say. 

I think  I would  go  all  the  way  back  to  the  Book  of  Exodus  for  docu- 

mentation of  this.  Moses  found  that  change  was  very  hard.  Even 
helping  the  ancient  Hebrew  slaves  change  to  a lifestyle  of  free 
persons  was  so  difficult  that  he  retreated  into  the  desert  for 
ho  years  and  waited  for  a new  generation  to  come  forth  that  could 
accept  freedom  a bit  better.  Eric  Hoffer  says  that  we  really  can't 
change  anyone,  except  through  the  generational  approach,  to  any 
effective  degree.  I might  also  refer  to  another  citation,  Romans 
5:17,  where  the  apostle  Paul  says,  "That  which  I would  do  I would 
not  and  that  which  I would  not  do  I do.  I find  the  will  to  perform 

but  how  to  perform  that  will  I find  not."  What  he  is  saying,  as 

I understand  it,  is  that  we  all  have  better  ways  of  doing  things-- 
all  of  the  users  certainly  don't  have  to  depend  upon  forthcoming 
research  findings  to  improve  their  practices.  But  there  is  some 
reason  that  we  don't  change  simply  out  of  the  awareness  of  a better 
way  to  do  i t . I am  fond  of  saying  that  anyone  who  thinks  being 
presented  with  a better  way  to  behave  is  sufficient  to  bring  about 
a change  in  behavior  has  never  been  married.  We  experience  a con- 
stant supply  of  free  consultation  on  how  we  can  improve  our  behavior, 
but  it  doesn't  really  do  a heck  of  a lot  of  good.  Well,  not  in  me 
anyway . 

What  can  we  do  about  it?  At  NIMH,  through  the  help  of  Dr.  Glaser 
and  other  people  who  have  been  tremendous  in  assisting  with  this, 
we  have  developed  an  eight  factor  approach  to  change  that  we  feel 
represents  the  necessary  and  sufficient  variables  that  must  be 
considered  if  we  want  utilization  of  knowledge  to  be  effective.  Let 
me  run  down  some  of  these  briefly.  If  you  should  want  to  read  about 
them  more  fully  you  will  find  a more  complete  discussion  in 
Dr.  Glaser's  manual.  Putting  Knowledge  to  Use.  Sometimes  when  people 
hear  us  talk  about  this  or  see  our  writings  on  this  they  say, 

"That's  a bother.  What  I want  to  know  is  how  I can  disseminate 
information.  I don't  want  to  get  into  organizational  change, 
program  change,  and  all  of  those  variables.  That's  simply  too 
complex."  All  I can  say  in  response  is  that  we  don't  have  to 

consider  all  of  these  variables,  but  if  we  don't  they  will  master 

us.  Our  only  choices  are  to  be  their  slaves  or  to  try  to  deal  with 
them.  They  are  not  there  by  our  invitation,  they  are  there  by  their 
own.  They  play  a role  in  all  instances  of  human  organizational 

change.  I find,  I might  also  say,  that  it  isn't  necessary  to  plan 

an  approach  to  handling  and  massaging  each  of  these  factors. 


The  technique  is  something  like  this:  the  model  is  never  mentioned 
to  the  people  with  whom  we're  working,  such  as  to  our  researchers. 

It  must  remain  in  the  back  of  our  heads,  though,  and  we  do  try  to 
make  observations  of  where  we  stand  on  each  of  these  variables. 

In  a formal  sense,  we  do  use  tests  to  determine  the  readiness  of 
an  idea  for  adoption.  Tom  Ki resuk  in  Minneapolis,  for  example, 
has  developed  a 120  item  scale,  using  factor  analysis,  that  allows 
one  to  predict  with  great  reliability  and  validity  whether  a 
change  is  going  to  come  a cropper  or  be  successful.  I find  that 
we  can  use  only  a seven  item  quas i -semant ic  differential  check  list 
and  get  a pretty  good  summary  of  how  people  are  feeling  about  these 
variables.  Then  what  you  do  is  develop  a profile  of  these  seven 
factors  and  not  try  to  deal  with  all  of  them.  You  might  find  that 
one,  say  res i stance , is  going  to  be  a particularly  tough  one  to 
overcome  and  then  one  works  only  on  that;  or  perhaps  it's  a 
mot i vat i ona 1 factor  and  so  we  work  only  on  that.  It  isn't  necessary 
to  deal  with  the  entire  array  of  factors.  There's  something  called 
the  Pareto  Principle.  Vilfredo  Pareto  was  an  Italian  economist  and 
sociologist  who  lived  about  the  turn  of  the  century,  and  his  assertion 
was  that,  in  instance  after  instance,  80  percent  of  human  achievement 
comes  out  of  about  20  percent  of  the  effort.  Now  what  we  need  to 
do  is  find  out  what  20  percent  needs  our  investment  and  we're  80 
percent  of  our  way  to  our  goal.  Well  this  is  the  way  the  scale 
operates.  We  try  to  figure  out  those  one  or  two  items  that  need 
massaging  and  work  on  those  only.  Not  only  does  this  system  help 
us  to  analyze  the  likelihood  of  successful  change  but  it  also 
gives  us  clues  as  to  what  we  can  do  to  massage  each  of  these  factors. 

Now  let  me  in  the  time  that  I have  remaining  briefly  tell  you  about 
a few  of  these  factors  just  for  illustration.  Certainly  one  is  the 
information  itself--how  well  is  it  disseminated,  how  practical  and 
cogent  is  it.  In  the  manual  that  I referred  you  to.  Dr.  Glaser 
also  has  an  acronym  that  he  calls  CORRECT.  It  covers  six  factors, 
six  attributes  of  information,  that  determine  to  a large  extent  how 
well  that  information  is  going  to  get  across,  what  its  impact  is 
going  to  be.  You  can  check  the  reference  to  find  out  what  those 
factors  are.  We  are  also  aware  of  factors  like  1 eg i t i mi zat i on . 

If  you  have  a new  device  or  an  innovation  that  you  want  some  command 
to  accept,  of  course  it  has  a greater  likelihood  of  being  used  if 
you  find  some  person  who  is  an  authority  who  will  put  an  imprimatur 
on  it,  a stamp  of  approval.  If  it  can  be  modeled  or  demonstrated 
by  someone  in  the  command  who  is  prestigious  and  an  opinion  leader 
in  your  particular  service,  then  of  course  it  has  a greater  like- 
lihood of  being  picked  up. 

In  terms  of  massaging  the  translation  of  information  we  can  draw  on 
the  old  work  of,  say,  Carl  Hovland  and  this  is  only  one  of  the  many, 
many  examples  that  one  could  draw  from.  If  you're  dealing  with  a 
sophisticated  group  such  as  this  one,  one  doesn't  try  to  present 
answers  as  to  exactly  how  you  do  it.  Sophisticated  people  are 
irritated  by  closed  gestalts  and  answers.  On  the  other  hand,  if 
you  are  presenting  information  to  people  who  do  not  consider 
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themselves  sophisticated,  such  as  the  typical  users,  they  are 
irritated  by  an  open  gestalt,  by  not  really  having  the  answers. 

They  want  training  manuals  with  things  explicitly  spelled  out  down 
to  the  last  word.  There  is  a whole  array  of  knowledge  from  the 

persuasion  field  such  as  Cohen's  or  Carlin  and  Ableson's  synthesis  ’ 

of  this  field.  Some  of  these  references  just  abound  with  ideas  that 
can  be  used  in  increasing  the  impact  of  the  transfer  of  information. 

Let's  move  on  to  another  one  of  the  factors,  mot i vat i on . Now  the  ; 

determination  that  there  is  a problem,  that  there  is  a need  to  do 
something  better,  is  okay,  but  that  in  itself  doesn't  change  us. 

I may  know  that  1 will  die  if  1 keep  smoking,  but  if  1 don't  feel 

the  need  to  change  I'm  not  going  to  do  much  about  it,  so  it's  felt  i 

need  that  really  matters.  Now  there  are  ways  that  a good  change 

agent,  a good  research  utilization  specialist,  can  increase  the  j 

felt  motivation  for  something.  For  instance,  once  in  Minnesota  we 

were  concerned  that  a small  psychological  services  team  that  was  ' 

operating  out  of  my  office  and  doing  psychological  tests  for  the  j 

public  school  system  around  the  state,  was  not  doing  the  job  ade-  i 

quately.  We  had  objective  data  to  show  that  we  couldn't  cover  the  \ 

task  but  none  of  the  local  school  districts  had  developed  their  own  i 

programs.  What  we  finally  did  was  to  cut  out  the  service  altogether 
so  they  had  no  one  coming  around  to  do  the  legally  required  testing, 
and  then  the  pain  came.  We  increased  the  obi i gat  ion , as  we  say  in 
this  particular  model,  or  the  mot i vat i on . They  had  to  do  something, 
so  throughout  the  system  local  school  psychology  programs  began  to 
be  developed.  Well,  there  is  much  we  can  do  to  increase  the  felt 
need  for  change  before  we  try  to  move  in  with  it. 

I think  c i rcumstances  and  timing  represent  terribly  important  varia- 
bles. Sometimes  it's  necessary  to  withhold  research  information 
until  its  time  has  come.  The  flux  of  things  may  be  just  right  for 

a ready  accommodation  of  it.  Right  now  if  they  were  to  wheel  in  j 

platters  of  food  probably  you  wouldn't  buy  it.  If  they  do  it  at 

6:00  this  evening  or  at  noon,  then  the  time  will  be  right  for  you 

to  transfer  that  commodity  into  your  own  bodies.  I think,  again 

referring  to  religious  history,  I would  like  to  recall  the  experience 

of  Martin  Luther  as  he  brought  about  the  Protestant  Reformation  in 

1532.  He  did  not  publish  in  some  obscure  theological  journal,  though  1 

they  existed  at  that  time;  instead  he  waited  until  the  priest  Tetzel 

had  been  going  through  the  German  communit'es  requiring  the  people 

to  pay  for  their  penances  and  the  poor  people  who  didn't  have  any 

money  couldn't  be  forgiven  for  their  sins  and  so  there  was  a great 

deal  of  unrest  growing  at  that  time.  But  he  waited  for  a special  I 

night,  and  that  was  Halloween,  the  eve  of  A1 1 Saint's  Day,  when  | 

people  gathered  in  the  small  town  squares  to  drive  evil  spirits  out  I 

with  effigies.  He  waited  until  the  crowd  was  in  the  town  square 

and  then  went  up  to  the  church  at  Vicksburg  and  nailed  his  95  theses  j 

on  the  wall  and  there  people  mobilized  around  him.  It  was  an  example  j 

of  exquisite  timing  on  the  part  of  a change  agent.  Whether  or  not 
you  agree  with  his  theological  concepts  we  must  agree  that  he  really 
demonstrated  how  a good  change  agent  can  operate. 


-79- 


f 


I 


There  are  other  factors,  such  as  how  to  measure  and  deal  with 
res i stance , how  to  increase  the  payoff , the  reinforcement  to  people. 
For  instance,  when  new  legislation  is  established  for  licensing 
for  a professional  group,  they  are  afraid  they  might  have  resistance 
from  the  older  people  who  don't  want  to  take  the  exams,  so  they  have 
the  technique  called  the  "grandfather"  clause  that  disarms  and  reduces 
the  resistance,  builds  a little  payoff,  reinforces  their  partici- 
pation in  it,  and  by  that  method  they  overcome  what  could  otherwise 
be  a great  obstruction. 

If  you  are  interested  in  reading  about  this  model  I think  you  might 
see  that  there  are  a number  of  ways  that  we  can  be  much  more 
instrumental  in  fostering  the  change  process  as  a result  of  the 
transfer  of  knowledge.  Now  as  to  how  we  build  this  into  management 
of  R&D,  well  that's  a question  that  everyone  has  to  answer  for 
himself  since  all  your  programs  vary  so  tremendously  much,  but  with 
the  kind  of  talent  that  I saw  here  yesterday  and  see  today,  I am 
sure  that  if  you  care  to  you  will  have  no  trouble  finding  a way  to 
achieve  the  very  important  mission  that  you're  involved  in.  Thanks 
very  much. 

DR.  GLASER:  Judah  Drob  has  been  in  charge  of  the  research  utilization 
area  for  the  Department  of  Labor  for  some  13  years  or  so,  and  we  will 
hear  from  him  now. 


DEPARTMENT  OF  LABOR  (POL) 


Mr.  Judah  Drob 

Chief,  Division  of  R&D  Utilization 
Office  of  Research  and  Development 
Department  of  Labor 


What  do  you  say  the  morning  after  the  bomb  has  been  dropped.  I thought 
about  what  it  was  that  I might  do  by  way  of  relief,  hopefully  some 
kind  of  rehabilitation  effort,  but  of  course  an  outsider  has  really 
very  little  right  to  say  very  much  about  your  internal  matters  and 
problems.  What  I've  heard  in  the  course  of  yesterday  and  today,  I 
found  to  be  really  very  heartening  and  significant  because  what  I 
heard  everybody  saying  is  that  utilization  is  everybody's  problem 
and  that's  exactly  right.  But  there  is  also  a danger  if  you  say 
that  utilization  is  everybody's  problem.  The  danger  is,  like  anything 
that's  everybody's,  it  turns  out  to  be  nobody's.  We  say,  for 
example,  that  public  parks  belong  to  everybody  and  in  fact  and  in 
practice  a lot  of  us  abuse  the  public  parks  because  we  feel  that  they 
belong  to  nobody.  Therefore,  I want  to  spend  a little  time  discussing 
with  you  the  opposite  side  of  the  coin--that  while  utilization  is 
clearly  everybody's  responsibility,  it  will  probably  not  be  adequately 
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done  unless  it  also  is  the  specific  responsibility  of  some  particular 
people.  Now  how  you  organize  that,  how  you  approach  it  within  your 
very  different  organizations,  is  obviously  up  to  you.  It  would  be 
very  unpolitic  and  would  fly  in  the  face  of  good  psychology  for 
anybody  to  prescribe  what  form  the  specific  responsibility  for 
utilization  should  take.  However,  even  though  there  is  some  danger 
in  doing  what  I am  about  to  do,  if  you  understand  that  I speak  to 
you  very  humbly,  I think  there  might  be  some  value  in  telling  you 
how  we  do  it  at  home.  Now  the  guy  who  shows  up  some  distance  from 
home  and  talks  about  "how  we  do  it  back  in  Toledo"  usually  gets 
short  shrift,  and  I don't  know  how  to  short  circuit  that.  But  I 
think  it  would  be  interesting  to  you  to  know  how  we  do  it,  as  long 
as  you  understand  that  I'm  not  telling  you  to  go  do  it  the  way  we 
do  it.  I'm  simply  trying  to  illustrate  one  way  that  this  might  be 
done.  You  very  likely  are  doing  it  better  already.  But  perhaps 
without  presenting  any  kind  of  model  I can  give  you  some  sense  of 
how  somebody  else  is  approaching  the  problem  and  maybe  that  will 
be  helpful.  If  it  isn't,  I apologize. 

I want  to  describe  to  you  what  I 1 i ke  to  think  of  as  a systematic 
approach  to  the  utilization  of  the  RSD  findings  and  products  that 
are  produced  for  the  Employment  and  Training  Administration  of  the 
Department  of  Labor.  We've  been  at  it  about  11  years.  We  started 
with  the  conviction  that  K^D  wouldn't  find  itself  into  use  just  by 
itself,  that  it  required  some  specific  effort.  Over  the  years  we 
have  developed  what  we  think  of  as  a system--one  that's  still  being 
built  and  still  has  some  gaps  in  it,  but  one  that  is  slowly  approach- 
ing what  we  would  like  it  to  be. 

Let's  start  at  the  beginning.  Mostly  what  we  do  is  fund  extramural 
research  and  development.  We  require  that  every  proposal  that  comes 
to  us  discuss  the  utilization  that  would  follow  successful  completion 
of  the  project.  In  the  discussions  and  negotiations  about  the 
funding  of  a particular  project,  we  require  consideration  about  what 
kinds  of  products  would  be  needed  in  order  to  utilize  what  might  be 
learned.  Obviously,  since  we  cover  a wide  range  of  kinds  of  R&D, 
from  pure  research  to  instrument  development,  the  nature  of  the 
products  will  vary  a lot.  But  in  the  case  of  research  that  might 
have  some  action  implication,  we're  looking  for  the  kinds  of  products 
that  can  be  used  in  the  field.  We're  talking  about  how-to-do-it 
mater i a 1 s--gu i del i nes  , manuals,  instructional  materials,  curricula, 
things  like  that--that  are  planned  to  be  produced  by  the  project 

in  the  course  of  its  life,  so  that  if  we  have  a good  project,  at  the 

end  we  will  have  materials  with  which  to  facilitate  utilization.  Nov/ 

saying  that  quickly  might  give  you  the  impression  that  it  is  easily 
done.  I have  yet  to  receive  a completely  satisfactory  product  out 
of  this  process  so  I don't  want  you  to  think  that  it's  easy.  Any 
time  that  I tell  you  how  great  we  are  I am  also  going  to  try  to 
mention  an  odd  little  thing  like  this.  The  consequence  is  that  we 
very  often  have  to  have  somebody  else  do  it  after  the  project  is 
completed.  We  very  often  have  to  spend  some  money  to  get  somebody 
to  put  the  results  into  usable  form  out  in  the  field. 
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During  the  life  of  some  projects,  particularly  those  that  are  not 
pure  research,  we  try  very  hard  to  involve  potential  users  on 
advisory  committees,  through  briefings,  through  site  visits  and  such 
activities  as  that,  with  the  aim  of  persuading  at  least  some 
representative  potential  users  that  the  product  is  really  their  product 
as  well  as  the  product  of  the  researcher  or  the  developer. 

When  the  project  is  completed  we  try  to  consider  all  possible  utili- 
zation strategies  that  might  be  appropriate  for  the  particular 
project.  And  for  this  purpose  we  have  invented  something  that  I think 
is  unique  in  the  utilization  field.  We  have  what  we  call  a "buddy 
system."  For  every  project  that  we  fund  a member  of  the  DOL  utilization 
staff  is  a utilization  buddy.  That  person  may  be  described  as  a 
conscience,  a consultant,  an  associate  project  officer,  or  whatever 
you  want  to  call  him.  That  person  has  the  responsibility  of  repre- 
senting the  utilization  concerns  and  working  on  the  utilization  plans 
for  the  project.  Now  let  me  say  right  here  that  the  best  project 
officers  are  the  ones  who  won't  let  go.  I don't  like  the  project 
officers  to  come  to  me  and  dump  the  project  on  me.  I want  them  to 
work  along  with  me  all  the  way  through  the  life  of  the  project  on 
utilization  plans.  Utilization  plans  might  include,  for  example, 
publications  for  external  audiences,  news  releases,  dissemination 
through  our  own  publications  for  our  own  universe  of  users,  and 
might  include  efforts  to  get  other  organizations  to  pick  up  where 
we  drop  the  project.  Obviously  this  is  not  an  all-inclusive  list. 
Utilization  efforts  such  as  these  are  less  likely  to  succeed  if  you 
come  along  after  the  project  is  completed.  Clearly,  the  best  results 
are  obtained  if,  during  the  life  of  the  project,  plans  are  built  into 
the  project  for  what  we  call  spin-off,  so  that  regular  programs  and 
operations  pick  up  what  we  have  developed. 

Now  I want  to  get  into  what  I think  for  us  is  the  heart  of  the 
problem.  The  line  from  R&D  to  the  academic  community  or  to  policy 
makers  is  pretty  direct.  I don't  think  R&D  has  much  influence  on 
policy  making,  but  that's  another  matter.  Anyway,  the  line  is  direct. 
But  when  you  talk  about  R6D  affecting  people  who  deliver  service,  or 
talk  about  the  armed  forces,  you're  talking  about  changing  the 
behavior  of  large  numbers  of  ordinary  folks,  and  obviously  you're 
talking  about  a training  program.  So  for  me  the  obvious  next  step 
in  the  system  of  knowledge  transfer  and  achievement  of  change  is 
the  training  and  technical  assistance  that  would  arise  out  of  the 
R&D  results.  Fortunately,  over  the  past  few  years  in  the  Employment 
and  Training  Administration  we  have  been  strengthening  our  technical 
assistance  and  training  capacity,  and  so  we  have  some  institutions 
in  place.  I want  to  stress  the  importance  of  this  kind  of  activity. 

It  doesn't  just  happen  that  you  have  a good  relationship  with  the 
organization  that's  next  in  line,  into  which  you  have  to  feed  your 
products.  I had  to  woo  the  people  in  charge,  and  the  consequence 
is  that  we  now  have  a good  relationship  with  them  and  an  opportunity 
to  feed  into  the  regional  training  institutes. 

Let  me  give  you  an  illustration  of  the  kind  of  result  that  we  like  to 
crow  about.  I had  funded  a project  to  experiment  with  linkages  with 


the  regional  training  institutes  to  improve  our  relationships  with 
those  institutes.  One  of  the  things  I'd  been  advocating  for  years 
is  something  called  a regional  resource  center,  which  perhaps  you 
can  imagine  without  my  describing  it  at  any  length.  I never  could 
get  any  regional  office  to  establish  one.  The  technical  assistance 
and  training  program  established  a clearing  house  by  which  it 
circulated  the  curricula  that  were  developed  in  all  the  ten  regional 
training  institutes,  the  theory  being  that  you  want  to  avoid  re- 
inventing the  wheel.  Well,  as  the  curricula  piled  up  in  all  the 
regional  offices  with  nobody  responsible  for  sorting  them  out, 
shelving  them,  indexing  them,  making  them  available  to  the  approp- 
riate people,  all  of  a sudden  the  light  dawned.  They  needed 
regional  resource  centers  for  their  own  products,  for  the  products 
for  which  I am  responsible,  and  for  the  products  of  state  and  local 
agencies  that  might  be  helpful  in  their  training  and  technical 
assistance  work.  So  all  of  a sudden  the  regional  resource  center 
became  a major  attraction.  And  my  contractor  was  picked  up  by 
the  technical  assistance  training  people  to  install  regional 
resource  centers  in  all  the  regions  on  a turnkey  basis. 

What  I'm  trying  to  illustrate  is  the  systematic  progress  that  one 
has  to  look  for  from  R5D  on  through  some  kind  of  connections.  In 
the  early  days  when  I first  met  td  Glaser,  Howie  Davis,  Bert  Griffis 
and  the  others,  we  were  trying  to  substitute  ourselves  for  all  the 
elements  of  this  system  of  knowledge  transfer.  We  were  trying  by 
ourselves  to  be  the  entire  system.  It's  very  difficult.  It 
probably  in  the  long  run  is  impossible,  but  by  beginning  to  get  the 
rest  of  the  organization  to  fit  in  and  to  set  up  its  own  appropriate 
agencies,  we're  starting  to  get  what  1 consider  to  be  the  beginnings 
of  a truly  systematic  approach. 

Now  there's  one  more  step  I have  been  working  on  a long  time.  I've 
raised  the  problem  often  with  Ron  Havelock  and  others.  I think  maybe 
I'm  beginning  to  get  some  products,  if  I can  get  some--pardon  the 
express ion--some  damn  slow  academicians  to  deliver  what  is  long 
overdue.  What  I'm  getting  at  is  the  problem  of  the  capability  of 
the  user  to  seek  out  information  and  to  use  it  to  solve  his  problems. 
It  doesn't  absolve  us  of  the  responsibility  for  doing  the  things 
that  we  have  to  do,  but  if  you're  broadcasting  and  nobody  has  a 
receiver  not  much  is  going  to  happen.  So  it's  important,  we  think, 
to  develop  the  capacity  of  the  ultimate  user  agency  and  individuals 
to  make  use  of  the  kinds  of  products  that  we  think  are  useful  to  them. 

This,  in  brief,  is  some  account  of  the  approach  we  take.  Now  I 
want  to  say  a word  about  philosophy.  You  are  probably  fed  up  to 
your  ears  with  the  notion  of  user  orientation  by  now.  I must  say 
to  you  that  user  orientation  is  the  first  commandment  for  us.  We 
feel,  however,  that  while  you  have  user  orientation  you  can  have 
your  own  dignity  and  rights  as  well,  and  that  it's  perfectly  proper 
to  tell  a potential  user  that  you  don't  have  anything  that  will 
help  his  particular  problem.  However,  we  try  not  to  do  that.  We 
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try  very  hard  to  help  people  solve  problems  and  if  we  don’t  have 
the  answer  we  try  to  help  them  find  places  where  they  can  get 
answers.  We  think  that's  very  important.  We  try  to  develop  their 
own  capacity  to  seek  information  and  to  find  answers.  I particularly 
want  to  commend  to  you  what  I have  seen  described  in  the  literature 
as  the  consultant  mode.  Now  the  consultant  mode  is  different  from 
a situation  in  which,  as  frequently  happens  of  course,  somebody 
comes  to  me  and  says,  "We're  going  to  do  something,  larumphing,  what 
do  you  know  about  larumphing?"  And  I go  to  the  shelves,  to  the  records, 
to  my  memory  and  we  draw  on  as  much  as  we  can  about  that  particular 
subject.  That's  relatively  easy.  The  consultant  mode  consists  of 
responding  to  a question  about  something  for  which  you  don't  have 
a direct  answer,  and  don't  have  exactly  the  right  materials,  but 
you  have  the  problem  of  extrapolating  from  what  you  know  to  the  some- 
what different  circumstances  about  which  those  people  are  asking 
you  questions.  1 think  that's  the  real  challenge  for  those  of 
us  in  the  utilization  business:  to  be  able  to  extrapolate  from 
what  we  think  we  know  to  the  real  and  current  problems  of  the  people 
that  we're  trying  to  serve.  I've  talked  long  enough  so  I'll  shut  up. 

Do  you  have  any  questions? 

DISCUSSION 

QUESTION:  At  one  point  you  indicated  that  you  generally  receive  cer- 
tain products  that  aren't  directly  useable  in  the  field,  so  you  have 
to  get  somebody  else  to  assist  in  refining  the  project.  What  kind  of 
agencies  do  you  use  for  this  purpose? 

MR.  DROB:  Well,  it's  kind  of  hit  or  miss.  We  try  to  find  people 
that  have  some  writing  or  curriculum  development  ability.  We  don't 
have  a large  stable  of  such  people.  Some  of  them  also  disappoint  us. 
Some  of  them  are  unreliable.  But  we  try  to  use  people  that  we  think 
know  enough  about  the  particular  subject  that  they're  writing  about 
to  be  able  to  do  a reasonably  good  job.  This  is  very  tough.  It's 
also  tough  under  procurement  regulations  which  were  designed  for  the 
purchase  of  hardware,  to  do  what  we  think  is  required  in  any  of  our 
contracting  and  granting  activities.  That's  a whole  different 
matter . 

DR.  GLASER;  Thank  you  Judah.  I think  Dr.  Meredith  Crawford  is 
probably  better  known  to  this  community  than  any  of  the  other 
panelists.  Before  coming  here  Meredith  has  been  head  of  HumRRO  for 
a number  of  years  and  now  is  at  George  Washington  University.  Meredith 
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For  many  years  when  I talked  about  HumRRO  my  remarks  were  carefully 
coordinated,  scrutinized,  and  criticized  by  my  senior  staff.  Two 


of  those  senior  staff  members  are  here  today.  You'll  hear  from 
the  current  president  of  HumRRO,  Bill  McClelland,  shortly.  You've 
already  heard  from  Bob  Smith.  These  gentlemen  may  be  surprised 
because  I have  not  coordinated  these  remarks  with  them. 

Perhaps  the  most  valuable  contribution  that  1 could  make  today  is 
to  bring  to  this  symposium  something  of  an  historical  note.  The 
observations  that  I will  make  come  from  my  own  notes  and  recollections 
about  the  first  17  years  of  the  existence  of  HumRRO,  that  is,  from 
1951  until  1967.  I choose  this  first  phase  principally  to  tell  you 
how  we  wrestled  with  some  of  the  problems  of  utilization  in  the  early 
days.  Also,  it  was  because  at  that  time  we  had  a single  sponsor, 
the  United  States  Army,  and  dedicated  all  of  our  efforts  to  that  one 
organization.  The  fact  that  our  research  was  utilized  accounts  for 
our  continued  presence.  The  reputation  that  we  had  for  a reasonable 
record  of  utilization  probably  accounts  for  why  I am  here  today. 

Now,  the  methods  that  we  developed  or  stumbled  onto  were  certainly 
in  part  a function  of  the  particular  circumstances  of  how  we  came  into 
being  and  how  we  grew.  I think,  though,  that  many  of  the  kinds  of 
general  principles  that  are  found  in  Ed  Glaser’s  blue  book  (Putting 
Knowledge  to  Use)  can  be  illustrated  by  the  things  we  did. 

Let  me  speak  just  briefly  about  the  climate  of  the  times  in  1951. 

There  was,  at  that  time,  a substantial  belief  in  the  country  that 
science  had  made  a contribution  to  our  victories  in  World  War  II. 

There  was  also  a reasonable  belief  that  psychologists  had  played  a 
useful  role  in  that  enterprise.  There  was  already  in  existence  in 
the  Army,  with  continuity  from  Vtorld  War  I,  research  in  the  person- 
nel branch  of  the  Adjutant  General's  office,  which  has  evolved  today 
into  the  Army  Institute  for  Research  in  the  Behavioral  and  Social 
Sciences.  There  was  also  a very  substantial  program  under  Dr.  Arthur 
Nelson  in  the  Air  Force.  At  that  time  money  was  perhaps  somewhat 
easier  to  obtain,  though  in  smaller  amounts,  than  is  the  case  today. 
Also,  there  had  been  a very  large  study  by  the  Defense  Research  Board 
of  the  Department  of  Defense  which  outlined  man-  new  things  that  could 
be  done  in  the  area  of  behavioral  and  social  s-  ence.  With  all  of 
these  background  activities  the  Army  was  ready  to  move  forward  in 
other  areas  of  research  in  human  factors.  I might  also  point  out 
that  while  the  climate  in  DoD  for  this  kind  of  activity  was  favorable, 
it  was  much  less  so  in  the  civilian  sector.  Massive  funding  for 
research  in  education  and  training  by  the  U.S.  Office  of  Education 
was  years  away.  The  American  Education  Research  Association 
numbered  a couple  of  hundred  members  as  compared  to  its  twelve 
thousand  members  today.  There  wasn't  much  precedent  for  applying 
research  and  development  to  military  training,  to  industrial 
training,  or  to  the  methods  and  curriculum  of  civilian  education. 

It  was  within  this  climate  that  Dr.  Harry  Harlow,  on  leave  from  the 
University  of  Wisconsin  to  serve  as  the  Army’s  Chief  of  Human 
Resources,  authored  a series  of  staff  studies  which  resulted  in  the 
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award  of  a sole-source  contract  to  the  George  Washington  University  for 
the  establishment  of  HumRRO.  The  new  organization  was  to  conduct 
research  in  training  methods,  motivation,  morale  and  leadership, 
and  psychological  warfare.  This  was  to  be  accomplished  by  research 
in  the  central  office,  by  subcontract  and  by  providing  civilian 
staff  technical  supervision  to  research  units  to  be  established  at 
military  installations. 

We  started  with  a broad  mission  and  with  adequate  funding.  What  we 
needed  was  a staff  that  could  begin  to  do  things  that  would  make 
sense  to  Army  people.  So,  making  sense  to  operational  military 
personnel,  especially  those  who  are  in  the  field  and  who  are  not 
directly  concerned  with  R6D,  was  an  early  task  as  we  set  about  the 
establishment  and  staffing  of  our  field  units,  the  first  at  Fort 
Knox  and  then  at  Fort  Ord.  In  order  to  justify  our  presence  at 
local  installations  and  to  receive  the  support  and  privilege  to  use 
the  troops  necessary  for  the  research,  we  had  to  propose  things  to 
do  that  made  sense  to  operational  people,  things  that  would  likely 
at  least,  yield  information  that  would  be  useful  in  improving  oper- 
ations, especially  training.  So  we  started  out  with  an  immediate 
orientation  toward  usefulness.  In  attempting  to  respond  to  this 
orientation  toward  usefulness  we  were  fortunate  in  building  our 
original  research  staff.  In  those  days  a large  portion  of  the  new 
PhDs  were  veterans  of  World  War  II,  many  with  combat  experience. 

Many  had  completed  their  education  under  the  G.l.  Bill  and  learned 
most  of  their  psychology  after  their  military  experience.  They  knew 
a good  deal  about  military  life  and  they  could  understand  the  skeptical 
view  of  the  "long-haired"  PhDs  that  were  held  by  many  operational 
people.  Thus,  for  the  most  part  the  initial  HumRRO  staff  had  a sense 
that  we  needed  to  produce  something  that  the  Army  could  use.  I 
should  note  that  very  early  in  the  game  I had  the  rule  that  there 
would  be  no  white  rats  at  the  Fort  Knox  laboratories  for  psychological 
research . 

So  my  first  and  most  fundamental  point  is  the  attitude  of  the  research 
people.  We  were  interested  in  doing  things  that  gave  promise  of 
being  useful  to  the  Army.  We  recognized  that  at  the  end  of  a re- 
search project  there  should  be  an  answer  to  the  question,  "So 
what?"  We  worked  at  developing  good  relations  with  our  military 
colleagues  and  we  found  them  both  professionally  and  personally 
rewarding.  We  learned  in  some  detail  about  military  operations  and 
came  to  recognize  the  type  of  military  scholarship  to  which  we  could 
relate  in  the  Army  schools,  in  the  Command  General  Staff  College 
and  in  the  War  College. 

This  attitude  of  interest  in  the  military  problem  and  the  desire  to 
become  involved  suited  most  of  the  people  who  stayed  at  HumRRO.  But 
it  took  awhile  to  clarify  our  thinking  about  the  nature  of  applied 
research  and  how  to  deal  with  practical,  rather  than  laboratory 
problems.  Some  hard  thought  was  done  by  those  who  had  recently 
come  from  the  academic  laboratory  and  this  took  place  over  the  first 
several  years.  Finally,  a clean-cut  summary  of  that  thinking  came 
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in  a chapter  written  by  Dr.  John  L.  Finan  in  Gagne's  Psycholog i ca 1 
Principles  of  Systems  Development.  Finan  compared  and  contrasted 
the  methods  appropriate  to  building  scientific  knowledge  through 
theoretically  oriented  inquiry  and  research  from  a practical  or 
engineering  point  of  view.  Of  the  eight  dimensions  along  which  he 
made  comparisons  I believe  that  three  influenced  our  thinking  most. 
The  first  concerned  the  criteria  of  acceptable  inference;  that  is, 
the  statistical  vs.  practical  genera  I i zabi I i ty  of  results,  which  is 
easy  with  a single-variable  laboratory  experiment  and  is  difficult 
with  a multivariable  practical  situation.  The  second  concerned 
the  differences  between  the  usefulness  of  demonstrating  a statistical 
significant  difference  for  purposes  of  testing  a well-stated  theory, 
and  the  demonstration  of  differences  which  are  large  enough  in 
magnitude  to  be  of  practical  or  what  we  now  call  cost-effective  use. 
The  third  concerned  the  difference  between  the  making  of  a prediction 
and  the  making  of  a forecast.  I take  time  to  mention  this  paper 
because  it  was  so  necessary  to  think  these  problems  through  in  the 
early  days  to  help  us  determine  the  criteria  of  good  applied  work. 
This  labor  made  us  "respectable"  with  our  "hasic"  colleagues  and 
helped  us  to  develop  appropriate  standards  of  quality.  We  were 
oriented  toward  carrying  our  work  through  to  an  ultimately  imple- 
mented solution,  and  having  thought  through  methodology  we  felt  sound 
in  doing  so. 


ly 


I might  add  that  we  were  not  immediately  perceived  as  useful  by  the 
Army.  At  the  end  of  the  second  year  1 went  before  the  R&D  review 
board,  a group  of  general  officers,  and  presented  what  we  wanted  to 
do  for  the  next  year.  And  the  chairman  said  to  me  very  simply,  "Well, 
doc,  we'll  give  you  money  for  one  more  year.  If  you  don't  produce, 
you're  out."  Fortunately,  TRAINFIRE  came  along  soon  after  that. 

In  addition  to  this  fundamental  observation  about  attitude  and 
orientation  toward  useful  results,  I want  to  mention  four  other 
factors,  all  of  which  may  be  found  in  the  knowledge  utilization 
literature,  that  1 think  were  important  in  the  development  of  HumRRO. 
They  are:  (1)  the  organizational  structure,  (2)  the  choice  of 
problems  and  methods,  (3)  some  administrative  categories  of  work 
which  facilitated  getting  to  useful  results,  and  finally,  (^)  some 
comments  on  the  presentation  of  results. 


With  respect  to  organization,  I have  already  mentioned  the  early 
establishment  of  field  units.  These  were  dual  organizations,  made 
up  of  the  research  personnel,  who  reported  to  the  director  of  HumRRO, 
and  a military  contingent,  headed  by  a field  grade  officer  who 
reported  to  the  commanding  general  of  CONARC , the  training  headquarters. 
The  mission  of  both  of  these  groups  was  the  same.  At  first  they  were 
confused,  later  they  were  clarified  on  an  organization  chart.  The 
enlisted  personnel  served  as  research  assistants,  one  of  whom, 

Dr.  Dick  Atkinson,  is  now  the  director  of  the  National  Science  Foundation. 
The  chief  and  the  director  of  research  of  each  field  unit  worked 
hand  in  hand,  representing  the  unit  at  the  local  command  in  briefings 
and  in  discussions,  and  along  with  the  researchers  briefed  on 
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research  at  the  high  headquarters.  We  found  it  was  most  desirable 
to  have  a chief  who  was  respected  in  his  branch  of  the  service 
(infantry,  artillery,  etc.),  a man  who  could  relate  to  his  peers 
in  that  branch.  The  divisions  in  the  central  office  of  HumRRO 
had  a less  immediate  relation  with  operations,  but  they  too  had 
thei r cl ientele. 

The  second  point  I would  like  to  make  in  the  context  of  organization 
is  that  our  relations  with  the  intermediate  training  headquarters, 
CONARC,  were  intimate,  early  and  fruitful.  We  were  fortunate  that 
our  first  staff  officer  (who  had  been  an  artillery  officer  in  World 
War  II  and  who  came  to  me  for  civilian  employment,  and  agreed  to 
go  back  on  active  duty)  was  superb  in  helping  the  first  two  years 
make  sense  to  those  at  Fort  Monroe.  Later  we  were  invited  by  the 
commanding  general  of  CONARC  to  place  a representative  there  and 
that  representative,  who  at  one  time  was  Bob  Smith,  took  every 
occasion  to  be  alert  to  what  was  going  on  in  the  headquarters  and  to 
the  need  for  bringing  researchers  to  the  headquarters  when  we  had 
relevant  information,  as  well  as  to  the  need  for  arranging  formal 
briefings  when  we  had  completed  phases  of  the  research.  The 
attitude  at  CONARC  was  favorable  in  this  sense.  I was  very  pleased 
once  when  a commanding  general  of  CONARC  said  to  me,  "We  cannot 
afford  to  ignore  a HumRRO  report."  Also,  when  our  relations  with  one 
officer  were  difficult  I went  to  the  commanding  general  to  point 
out  that  many  times  in  order  to  do  our  work  it  was  necessary  to  seem 
to  criticize  the  Army  in  a positive  and  constructive  manner.  The 
general  understood  this  much  better  than  the  colonel,  which  is 
another  story. 

Another  point  in  connection  with  the  organization  is  that  our 
relations  at  the  Department  of  the  Army  were  with  the  Chief  of 
Research  and  Development.  This,  I always  felt,  was  an  extremely 
important  relationship,  particularly  when  it  was  coupled  with  that 
at  CONARC,  because  it  made  us  part  of  the  larger  research  and 
development  system.  We  had  a model,  a hardware  model  that  we  all 
know  doesn't  fit  exactly,  but  it  was  a useful  point  of  reference  and 
we  were  with  colleagues  who  were  concerned  with  doing  new  things 
within  the  R&D  mechanism.  The  combined  relationship  to  Headquarters, 
Department  of  the  Army  R&D  and  to  an  operational  headquarters  at 
CONARC  was  very  beneficial.  We  used  to  speak  of  the  triumvirate 
relationship  between  the  Department  of  the  Army,  CONARC  and  the 
Director  of  HumRRO.  It  had  its  problems  but  it  paid  off  in  good 
commun i ca  t i on . 

Finally,  I would  make  one  other  point  with  respect  to  organization. 

We  were  fortunate  in  being  able  to  have  a dispersed  organization 
and  yet  one  with  substantial  communication  across  the  organizaton 
so  that  things  which  developed  in  one  part  of  the  organization  could 
be  shared  in  another  setting.  There  was  a "not  invented  here" 
phenomenon  with  which  we  are  all  familiar.  But  there  were  occasions 
where  one  division,  say  the  infantry  division  at  Fort  Benning,  came 
up  with  something  and  through  our  organization  and  our  understanding 
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of  problems,  say  in  air  defense,  that  solution  could  be  moved 
over.  We  scent  a good  deal  of  time  in  conferences,  sometimes 
with  our  military  chiefs,  sometimes  as  directors  of  research,  in 
discussing  our  methods,  in  discussing  how  we  could  relate  to 
the  Army,  and  how  we  could  make  our  combined  impact  felt  more 
keenly. 

The  second  observation  1 will  talk  about  concerns  our  choice  of 
problems,  some  of  which  turned  out  to  be  nutritious  objects  of 
; attention.  Later,  at  our  suggestion,  the  "dragnet"  started.  That 

is,  each  year,  the  CONARC  headquarters  began  to  send  inquiries 
throughout  the  command  with  respect  to  problems.  Now,  as  has 
been  discussed  here  today,  often  these  problem  statements  are 
not  ready  to  be  undertaken  as  research  projects.  We  found 
, that  if  it  was  possible  for  us  to  go  to  the  originator  of  the 

problem  and  talk  with  him,  the  problem  would  be  much  more  clearly 
defined  and  perhaps  it  was  somewhat  different  from  what  he  per- 
ceived it  to  be.  But  the  dragnet,  which  developed  into  a 
requirements  survey,  was  an  extremely  useful  way  of  getting 
broad  coverage  and  of  combing  various  kinds  of  problem  statements 
P into  a single  investigation,  the  results  of  which  might  have 

some  genera  1 i ty . 

With  respect  to  method  of  research,  insofar  as  possible  we  relied 
on  the  experimental  method.  I remember  way  back  in  1952  going 
down  to  Fort  Monroe  and  lecturing  with  a big  chart  on  the  experi- 
mental method,  talking  about  the  experimental  group  and  the  control 
group,  and  some  of  my  staff  thought  that  didn't  make  much  sense, 
but  it  "took"  and  within  a year  people  were  asking  "Well,  where's 
your  control?"  The  sharing  of  what  we  were  trying  to  do  and 
how  we  were  trying  to  do  it  was  extremely  useful  and  more 
importantly  we  were  able,  in  most  instances,  to  demonstrate  with 
hard  data  that  this  technique  yielded  better  results  than  that. 

A third  point  1 want  to  make  is  about  the  flexibility  of  our  work 
program,  particularly  the  categories  of  work  in  which  we  operated. 

We  began  with  nothing  but  tasks,  which  1 believe  we  call  work 
, units  now.  These  were  rather  long  efforts  involving  several 

people.  We  introduced  a new  category  called  exploratory  studies, 
in  which  we  would  take  a problem  and  not  more  than  one  or  two 
research  personnel  would  stay  at  it  for  a matter  of  months.  Either 
we  would  determine  that  we  weren't  getting  anywhere  or  that,  with 
apologies  to  an  earlier  speaker,  there  could  be  an  administrative 
solution,  or  that  it  was  worth  a full  scale  effort.  Then  we  would 
turn  it  into  a task.  About  50  percent  of  these  exploratory  studies 
were  abandoned  and  it  was  understood  that  that  was  the  expectation. 

As  1 told  General  Gavin,  Chief  of  Research  and  Development,  a good 
R6D  program  has  about  50  percent  duds.  But  we  provided  for  attempting 
to  identify  those  duds  as  early  as  possible  and  getting  on  with  the 
workable  problems. 

Another  aspect  of  flexibility  is  that  we  made  an  effort  to  respond 
to  inquiries  at  the  local  installation  or  elsewhere,  like  "What 
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do  you  know  about  so  and  so;  is  there  any  research  data  on  this?" 

We  took  time  to  do  that  because  we  wanted  to  be  useful.  We  also 
wanted  to  learn  more  about  the  Army  and  responding  to  these  inquiries 
helped  us  do  that.  Eventually  we  also  got  into  the  work  program  a 
category  of  technical  advisory  service,  TAS  as  it  was  called,  and 
we  engaged  in  some  100  or  more  TAS  activities  each  year.  These 
were  important  not  only  in  furnishing  information  but  in  enabling 
us,  even  though  we  were  funded  with  6.2  money,  to  help  write  training 
manuals  or  help  do  whatever  was  necessary  to  get  a project  or  product 
implemented.  I said  earlier  that  we  concerned  ourselves  with  the  kind 
of  work  we  did  and  I remember  that  at  the  director's  meeting  one  day 
we  discussed  with  great  care  the  circumstances  under  which  we  would 
give  TAS,  because  we  were  concerned  about  shooting  from  the  hip, 
and  we  wrote  a rather  specific  directive  to  ourselves  that  when  the 
problem  was  sufficiently  important  and  when  we  felt  we  had  a prob- 
ability of  slightly  better  than  .5  of  giving  a useful  answer,  we 
would  do  so.  1 emphasize  that  we  were  concerned  on  the  one  hand  to 
be  helpful,  but  we  were  equally  concerned  that  we  not  go  off  half 
cocked  or  give  misleading  information. 

The  final  point  1 want  to  talk  just  a little  bit  about  is  the  pre- 
sentation of  results.  We,  like  all  other  organizations,  struggled 
with  such  issues  as  what  a technical  report  should  be,  to  whom  it 
should  be  directed,  and  whether  we  make  recommendations  or  draw 
conclusions.  We  spent  a great  deal  of  time  on  the  quality  of 
technical  reports.  Fortunately,  in  those  days  we  were  not  bound  by- 
contract  deadlines  so  we  sometimes  took  several  months  to  get  a good 
report,  but  we  also  found  that  once  that  report  was  in  the  literature 
it  was  often  used  by  the  Army,  in  some  cases  years  later.  Reports, 
of  course,  were  not  enough.  Briefings  were  essential,  and  having 
the  military  personnel  who  were  involved  in  the  research  (not  only 
our  chiefs  but  the  user)  take  part  in  these  briefings  was  most  con- 
vincing. We  did  a little  project  for  the  Navy  and  when  the  research 
leader  briefed  the  admirals  he  said  very  little  himself.  He  had  the 
chief  petty  officers  do  the  briefing.  That  was  substantially  more 
convincing  in  that  case  than  doing  it  himself. 

One  other  point  worth  mentioning  is  that  the  Chief  of  Research  and 
Development  sent  a letter  of  transmittal  with  all  our  technical 
reports.  We  usually  drafted  that  letter  but  the  letter  attempted 
to  show  ways  in  which  the  report  would  be  useful,  and  in  particular, 
to  whom  it  was  directed.  In  addition  to  our  concern  with  the 
presentation  of  our  own  results  we  were  fortunate  that  for  a number 
of  years  the  CONARC  staff  prepared  what  was  called  a utilization 
pamphlet,  a regular  CONARC  publication,  in  which  all  of  the  instances 
of  utilization  of  HumRRO  work  were  reported,  and  this  was  widely 
distributed  throughout  the  land.  It  often  resulted  in  requests  to 
us  for  reports  of  some  of  these  pieces  of  work  and  in  a good  deal  of 
TAS.  We  also  blew  our  own  horn.  We  published  a thing  we  called 
"What  HumRRO  is  Doing,"  first  as  a sort  of  illustrated  annual  report 
and  again  as  a newsletter.  We  did  believe  in  persistent,  but  I hope 
dignified  and  effective,  selling.  We  liked  the  Army.  We  enjoyed 
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working  in  the  Army.  We  found  our  contacts  stimulating,  in  many 
cases  scholarly,  and  we  felt  that  we  were  part  of  the  team. 

I think  that  in  summary  what  I would  like  to  bring  from  the  historical 
perspective  is  first,  the  attitude  of  our  research  people.  Our 
paramount  interest  was  in  doing  high-quality  work  but  with  the  needs 
of  the  user  understood  as  completely  as  we  could.  Second,  we  had  an 
organization  which  was  dispersed  but  yet  communication  within  it 
was  good  so  that  we  could  understand  and  communicate  widely  in  the 
Army.  Third,  we  had  a flexible  work  program  that  was  adapted  to  a 
variety  of  short-term  and  long-term  needs.  Finally,  we  took  sub- 
stantial care  in  the  various  forms  by  which  we  presented  our  results. 
Well,  that's  the  way  it  looks  to  an  old  man  who  may  like  to  talk 
about  the  way  things  used  to  be,  but  I think  there  are  some  lessons 
there. 

DR.  GLASER:  Our  next  speaker  is  Bert  Griffis,  who  is  representing 
the  Rehabilitation  Services  Administration. 

REHABILITATION  SERVICES  ADi^INISTRATION  (RSA) 


t-tr.  Bertnon  W.  Griffis 
Assistant  Chief 
Division  of  Program  Support 
Rehabilitation  Services  Administration 


"What  a piece  of  work  is  man."  In  that  one  sentence  I do  believe 
Shakespeare  summarized  quite  a few  of  our  problems.  I perceive,  as 
you  perceive,  that  I am  still  somewhat  under  the  influence  of  having 
seen  Hamlet  at  the  Globe  theatre  Sunday  evening  and  I do  hope  you 
will  bear  with  me.  Short  though  my  time  may  be,  I cannot  refrain 
from  bringing  greetings  from  Virginia,  home  of  several  of  our  leading 
founding  fathers  who  utilized  some  of  the  simplest  but  most  important 
lessons  of  history  in  framing  our  Constitution,  and  thereby  produced 
a document  that  has  been  called  one  of  the  greatest  ever  to  spring 
from  the  mind  and  pen  of  man.  And  I think  we  can  conclude  from 
this  happy  event  that  the  future  of  utilization  is  immense  and  up- 
lifting, especially  when  the  findings  that  are  put  to  use  have  been 
tested  in  the  laboratory  of  history  as  Will  and  Ariel  Durant  put  it 
in  their  little  book.  The  Lessons  of  History. 

Now  I would  also  bring  greetings  from  Washington  if  I knew  what  to 
say,  but  I don't  for  doth  not  yet  appear  what  shall  transpire,  because 
one  of  the  innovations  adopted  by  the  founding  fathers  was  that  of  a 
system  of  checks  and  balances  which  they  wove  into  the  constitution, 
their  intent  being  to  prevent  rule  by  any  one  man,  party  or  philosophy, 
to  facilitate  the  rule  of  law  rather  than  of  men.  And  the  result 
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is  that  we  do  not  really  know  what  is  going  to  happen.  We  do  not 
know,  we  cannot  know,  what  the  net  outcome  of  those  checks  and 
balances  is  going  to  be.  So,  then,  the  founding  fathers  created  a 
paradoxical  situation.  Mainly  by  utilizing  a particular  bit  of 
historical  wisdom  they  made  it  impossible  to  predict  precisely  or 
even  approximately  what  may  happen  in  a given  instance,  and  that, 

I submit,  is  in  accord  with  the  spirit  of  liberty  which,  as  Judge 
Learned  Hand  has  noted,  is  the  spirit  that  is  not  quite  sure  it's 
right.  Whereas  the  spirit  of  tyranny--and  I do  think  it  permissible 
to  speak  of  the  spirit  of  tyranny  in  1977,  seeing  that  quite  a few 
tyrannies  do  exist — the  spirit  of  tyranny,  I say,  is  always  sure 
it's  right  and  always  knows  exactly  what  is  going  to  happen;  I mean 
what  it  will  permit  to  occur.  A tentative  conclusion,  if  conclusions 
may  be  drawn  from  an  introduction,  may  be  that  with  respect  to  certain 
deep,  enduring,  recurrent  and  really  important  human  problems,  the 
humanities,  history,  philosophy,  literature,  and  the  fine  arts  may 
have  more  to  offer  than  RSD  in  the  usual  sense  of  the  word.  Well, 

I have  drawn  conclusions  from  an  introduction.  But  please  bear  in 
mind  the  influence  I am  under.  Hamlet  behaved  erratically,  but  in 
the  end  got  the  Job  done,  after  a fashion. 

I represent  the  Rehabilitation  Services  Administration  in  the  depart- 
ment of  HEW  and  we  manage  a nationwide  state  federal  program  of 
rehabilitation  for  disabled  people,  funded  overall  at  an  annual  level 
of  about  800  to  900  million  dollars,  with  most  of  the  money  going 
for  the  multiplicity  of  services  needed  by  handicapped  clients  in 
the  field.  Our  mission  is  to  help  as  many  of  these  clients  as  we 
can  to  become  as  independent  and  self-sustaining  as  possible.  Our 
emphasis  has  been  vocational  with  more  attention  lately  to  the 
severely  handicapped.  As  a part  of  all  this  we  fund  an  R&D  program 
of  about  30  million  dollars  annually,  plus  a program  of  about  the 
same  level  to  train  professionals  needed  to  work  in  this  interdis- 
ciplinary field  of  rehabilitation. 

Our  utilization  program  began  in  1966  with  a memo  directing  us  to 
identify  usable  R6D  results  and  establish  ways  and  means  to  get  them 
used  in  service  programs.  From  this  has  grown  a modest  research 
utilization  program  with  the  following  broad  goals:  (l)  to  plan, 
conduct  and  report  research  so  as  to  maximize  the  likelihood  of 
usable  results  being  produced,  (2)  to  identify,  package  and  dissemi- 
nate usable  results  to  consumers  in  the  field,  (3)  to  establish  human 
and  organizational  links  in  the  field  between  research  and  practice, 

(4)  to  generate  or  otherwise  acquire  knowledge  on  research  utilization 
and  the  diffusion  and  adoption  of  innovations,  (5)  to  build  into 
service  agencies  a receptivity  to  change  and  a capacity  for  ongoing 
self-renewal,  and  (6)  to  evaluate  the  impact  of  these  research  utili- 
zation efforts. 

Now,  we  have  really  spent  very  little  money,  only  a few  million  dollars, 
I believe,  over  several  years,  and  in  this  regard  we  feel  quite  good. 

Our  hands  are  very  clean.  Though  we  have  spent  little,  we  believe 
we  have  accomplished  much,  namely: 


-92- 


!1 

(i 


ft 


1.  We  have  abstracted  and  indexed  all  R&O  reports  received.  These  ; j 

are  substantive  abstracts  with  the  use-potential  section.  We  1 

have  published  those  abstracts  along  with  a general  index  and  i 

an  RSD  thesaurus,  and  distributed  these  documents  to  strategic 

users  in  the  field.  And  we  have  also  published  a series  of 

research  briefs  and  disseminated  each  one  to  as  many  as  30,000 

users.  ’ 

f 

2.  We  have  entered  all  R£D  reports  into  the  National  Technical  j 

Information  Service  (NTIS)  to  give  us  storage,  retrieval,  I 

microfiche  and  hard  copy  capability.  The  reports  are  in  NTIS;  I 

some  people  have  told  us  you  can't  get  them  out,  but  they  are  j 

in  there.  | 

3.  We  have  published  and  disseminated  a guide  for  preparing  final  | 

R6D  reports,  emphasizing  utilization  factors,  calling  for  a 

document  that  will  facilitate  use  of  results,  and  requiring  a 
utilization  plan  (assuming  there  is  a usable  result)  that 
identifies  the  usable  product,  the  clients  and  problems  to 
which  it  applies,  the  resources  needed  to  use  it,  its  limita- 
tions and  the  cautions  to  be  observed  in  using  it,  and  the  j 

expected  benefits  from  its  use. 

k.  We  have  evaluated  a large,  random  sample  of  our  completed  R6D 
reports  and  thereby  derived  findings  and  principles  to  make 
them  more  productive. 

5.  Using  these  and  other  data,  we  have  specified  criteria  for 
evaluating  R&D  proposals  (and  here  we  are  also  greatly  indebted 

to  Dr.  Glaser  and  to  Howard  Davis),  setting  forth  in  some  de-  j 

tail  desirable  utilization  elements  that  a proposal  should  1 

embody;  and  we  have  disseminated  these  criteria  as  one  part  j 

of  our  application  kit.  I 

I 

6.  V/e  have  conducted  and  evaluated  a national  demonstration  pro-  i 

gram--a  five-year  program  of  agents  in  state  rehabilitation  ' 

service  agencies — to  link  practice  with  research,  somewhat  j 

along  the  line  of  the  county  agent  in  the  Department  of  I 

Agriculture.  We  are  greatly  indebted  to  Dr.  Glaser  for  his 

evaluation  of  this  program,  for  conducting  a utilization 

conference  on  it  to  which  were  invited  all  directors  of  state  I 

rehabilitation  agencies  and  power  figures,  and  for  preparing  ■ 

guidelines  for  installing  these  change  agents  in  state  service 
agencies. 

7.  We  have  contracted  to  have  designed  for  our  agency  alternative 
utilization  systems  that  might  serve  our  needs  more  compre- 
hensively, the  powers  that  be  permitting  (which  they  have  not 
up  to  now).  We  are  much  indebted  to  Dr.  Ron  Havelock  for  this 
most  useful  piece  of  work. 

8.  We  are  funding  two  research  utilization  laboratories  to  serve,  j 

on  a regional  basis,  as  institutional  links  in  the  field  between  j 

I 
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research  and  practice,  and  haye  partially  evaluated  the  work 
of  one  of  them. 

9.  We  have  instituted  annual  participatory  planning  conferences  to 
feed  real  operating  problems  into  our  R&D  planning  hopper, 

10.  We  fund  six  or  eight  rehabilitation  engineering  centers,  primaril 
to  develop  hardware  to  be  used  to  help  the  handicapped,  and  to 

be  used  by  the  handicapped  to  help  themselves.  These  centers 
are  currently,  in  addition  to  the  other  work,  preparing  state- 
of-the-art  papers  on  10  hardware  areas  relevant  to  improving 
services  for  the  handicapped. 

11.  We  have  infused  a strong  R£D  utilization  element  into  our 
$30,000,000  program  to  train  a variety  of  professionals  needed 
to  serve  the  handicapped. 

Those  are  some  of  the  main  things  we  have  done.  Now,  in  the  course 
of  this  and  other  work  not  mentioned,  we  have  become  conscious  of 
certain  problems,  of  which  1 will  mention  several. 

1.  Science  cannot  resolve  conflicts  of  interest,  cannot  be  made 

to  yield  value  judgments.  This  is  our  opinion. 

2.  We  have  become  mistrustful  of  research  utilization  (RU)  insofar 

as  it  counts  its  chickens  before  they  are  hatched.  1 myself 
have  been  involved  in  quite  a few  activities  to  count  these 
chickens  rather  accurately  before  they  are  hatched.  Now  it 

is  true  that  chickens  do  hatch,  but  sometimes  they  are  birds 
of  a strange  feather. 

3.  It  is  idle  to  hope  that  our  troubles  as  people  can  be  cured 

by  any  amount  of  knowledge  or  knowledge  utilization  per  se.  We 

seem  to  want  knowledge  in  RU  to  accomplish  painlessly  for  us 
what  can  in  fact  come  only  from  quite  another  quarter.  We 
doubt  if  lack  of  scientific  knowledge  is  always  the  main  problem. 

There  are  three  great  sources  of  reliable  knowledge:  science; 
the  humanities,  which  dramatize  and  summarize  human  experience; 
and  the  intuitions  of  great  ethical  and  moral  geniuses.  Now, 
with  respect  to  science,  a sure  prescription  for  quackery  is  to 
succumb  to  such  externally  imposed  mandates  as  those  from  below 
by  students--for  "relevance,"  and  those  from  above  by  funding 
agenci es--for  preconceived  responsiveness.  Whatever  responds 
primarily  to  such  mandates,  ought  not  to  call  itself  science. 

5.  It  is  very  hard,  if  not  impossible,  to  ensure  that  RU  benefits 
trickle  down  to  the  ultimate  clients  (in  the  case  of  our 
agency,  handicapped  individuals)  that  are  most  in  need  of  help. 

In  other  words,  it  is  hard  to  prevent  RU  from  becoming  an  end 
in  itself.  That  is  most  unfortunate  for  many  of  us--not  only 
the  hand i capped--who  see  our  actual  lives  as  deformations  of 


our  possible  lives.  And  as  Ortega  has  noted,  "every  life  is 
more  or  less  a ruin  among  whose  debris  we  have  to  search  to 
discover  what  it  might  have  been." 

6.  RU  has  a tendency  to  contri bute  to  the  information  explosion, 
rather  than  to  respond  constructively  to  it. 

7.  In  a world  so  full  of  change  and  innovation,  it  is  almost 
impossible  to  know  what  causes  what,  very  hard  I mean  to  evaluate 
RU  results.  Nevertheless,  I am  optimistic,  for  as  the  poet  has 
written,  "Hope  burgeons  in  the  soul  and  sings  and  never  stops 

at  all".  Thank  you. 

DR.  GLASER:  Ron  Havelock  has  probably  written  more  on  research  utili- 
zation, especially  related  to  education,  but  not  by  any  means  limited 
to  that,  than  perhaps  anybody  in  the  country.  Ron  is  from  the 
University  of  Michigan  and  the  Institute  of  Social  Research  within 
which  there  is  an  organization  known  by  the  acronym  of  CRUSK,  which 
stands  for  the  Center  for  Research  Utilization  of  Scientific  Knowledge. 
Ron  will  tell  you  about  the  program  of  the  National  Institute  of 
Education  in  the  area  of  research  utilization. 


NATIOriAL  INSTITUTE  OF  EDUCATION  (NIE) 


Dr.  Ronald  G.  Havelock 
Program  Director 

Center  for  Research  on  Utilization  of  Scientific  Knowledge 
Institute  for  Social  Research 
The  University  of  Michigan 


When  we  talk  about  research  utilization  we  can  talk  about  it  as  a 
process  of  something  that  happens,  but  we  can  also  talk  about  it  as  a 
system,  and  also  as  an  institution.  Somewhat  buried  at  times  is  the 
fact  that  these  speakers  are  talking  about  institutions  which  specialize 
in  research  utilization  and,  as  I understand  it,  none  of  the  military 
services  at  the  present  time  actually  has  an  institution  or  set  of 
people  who  are  specifically  and  exclusively  responsible  for  research 
utilization.  Now  that  is  quite  distinct  from  many  of  the  speakers 
that  have  preceded  me.  In  Meredith  Crawford's  speech  we  get  an  idea, 
through  HumRRO,  of  how  an  institution  can  emerge  that  in  effect 
plays  a utilization  function.  The  distinction  would  be  that  HumRRO 
represents  an  organization  that  was  concerned,  and  continuously 
concerned  (in  part  because  of  the  style  of  leadership  and  motivation 
of  its  own  leadership)  with  research  utilization,  but  did  not  devote 
itself  exclusively  to  that  task,  nor  perhaps  was  there  even  room  or 
permission  to  do  so. 
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Well,  coming  at  the  end  of  these  speakers,  I want  to  say  something 
about  what  has  happened  in  the  education  field,  because  probably  we 
have  had  more  deliberate  effort  there  to  set  up  institutions  for 
research  utilization  on  a fairly  large  scale  than  in  any  of  the  other 
examples  we  have  heard.  The  Agriculture  Cooperative  Extension  Service, 
which  was  previously  mentioned,  is  by  far  the  largest  and  oldest 
research  and  utilization  system  in  the  world  and  it  probably  is  the 
best  working.  Indeed,  we  could  say  we  are  a bit  remiss  in  not  having 
somebody  on  the  program  to  describe  it,  because  the  experience  of 
that  program  is  extremely  helpful  to  the  problem  of  research  utili- 
zation in  the  military.  We  have  not  talked  about  how  we  can  make 
use  of  all  the  rest  of  the  R&D  that  is  available,  the  wisdom  of  the 
ages,  not  just  of  recent  times.  No  one  has  mentioned  the  Defense 
Document  Center,  which  is  interesting. 

When  they  decided  to  do  something  about  research  utilization  in  the 
field  of  education,  RSD  in  most  consumer  sectors  was  not  popular, 
until  maybe  in  the  1960's.  But  there  was  an  accumulation  of  research 
information  under  the  Cooperative  Research  Act  and  National  Defense 
Education  Act,  stimulated  in  part  by  competition  with  the  Russians 
to  build  a research  program  in  education.  Our  national  investment 
in  education  is  probably  the  largest  of  any  particular  categorical 
sector  in  the  entire  economy,  much  larger  than  the  military.  But  in 
terms  of  R&D  it  is  probably  the  smallest  area,  in  the  sense  of  the 
ratio  of  R&D  investment  to  total  investment.  Nevertheless,  over  the 
years  starting  in  the  early  to  mid-1950's,  probably  as  much  as  one- 
and-a-half  or  two  billion  dollars  worth  of  research  has  accumulated. 
Indeed,  there  are  people  who  carry  on  research  on  a small  scale  in 
various  settings  all  across  the  country  in  education,  mostly  in 
schools  of  education.  The  first  step  towards  making  an  effort  to 
utilize  this  research  was  the  creation  of  a thing  called  ERIC-- 
Educational  Resources  Information  Center,  really  centers.  It  was 
an  attempt  to  apply  all  the  technology  of  information  retrieval  in 
the  early  1960's  to  educational  R&D  products.  The  idea  was  to  have 
a system  that  would  enable  any  research  utilization  activities  to  tap 
into  all  the  existing  research  that  was  available  in  the  field  of 
education,  not  just  what  happened  to  be  in  the  head  of  a particular 
person  at  a particular  time. 

I entered  the  act  in  about  1967.  One  of  the  first  projects  I had 
in  research  utilization  was  to  look  at  what  could  be  done  to  create 
a system,  in  addition  to  ERIC,  that  really  made  a connection  between 
information  and  use  and  between  the  R&D  community  and  the  user 
communities.  Out  of  the  thinking  about  what  we  could  do  in  addition 
to  ERIC  gradually  emerged  what  might  loosely  be  called  a research 
utilization  system  for  education.  Today  it  is  part  of  the  legislative 
mandate  of  the  National  Institute  of  Education  (NIE)  to  provide 
dissemination  services.  Out  of  the  total  budget  of  70  million  dollars 
for  R&D  in  the  National  Institute,  17  million  is  devoted  to  dissemina- 
tion activities,  including  about  5 million  dollars  which  supports 
the  ERIC  system.  That's  a pretty  good  ratio  of  funds  devoted  to 
research  utilization  out  of  the  total  research  budget.  I'm  sure 
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I that  it's  by  far  the  largest,  proportionally,  of  any  agency.  So 

what  they  do  is  worth  looking  at  and  is  an  experience  that  I think 
the  people  in  the  military  who  are  concerned  about  setting  up 
institutional  mechanisms  for  research  utilization  really  ought  to 
look  at. 

NIE  has  experimented  with  a number  of  kinds  of  activities  that  are 
similar  to  what  you've  talked  about  here.  One  of  the  first  things 
that  they  realized  they  had  to  do  with  ERIC  was  to  engage  in  a great 
deal  of  transformation  activity.  And  indeed  there  are  innumerabie 
kinds  of  transformations  you  could  think  of,  such  as:  summarization, 
abstracting,  transforming  for  particular  user  groups,  transforming 
from  a print  medium  to  audiovisuai  presentations,  taking  documents 
^ that  are  in  research  language  and  translating  them  into  practitioner- 

type  language,  etc.  For  at  least  ten  years  a certain  portion  of  the 
budget  has  systematically  been  allocated  to  transformation  functions. 

Beyond  transformation,  however,  it  was  obvious  that  connections  were 
needed  of  what  one  might  call  a more  human  nature,  and  again  they 
iooked  down  the  street  to  what  the  Cooperative  Extension  Service  was 
up  to,  or  had  been  up  to  for  years:  the  much  talked  about  but 
actually  very  little  studied  county  agent  notion.  And  they  decided 
that  they  should  try  to  do  a little  experiment  to  see  whether  you 
' couid  have  a county  agent  in  education  who  might  work  in  a similar 

kind  of  way.  Institutional  realities  being  what  they  were,  they 
thought  they  wouid  set  these  up  in  state  departments  of  education. 
They  did  so  in  three  states  in  the  early  1970's.  It  was  a relatively 
small  program,  a pilot  project  really,  to  see  what  might  work  and 
I whether  it  might  be  something  that  could  blossom  into  a rather  large 

I program  nation-wide.  Not  only  did  they  install  these  agents  but 

I they  compared  them.  They  arranged  to  have  them  studied  by  a fairiy 

[ distinguished  sociologist  who  was  quantitative  in  his  orientation. 

I The  results  of  the  study  were  that  this  speciai  linking  agent  was 

j very  much  appreciated  by  locai  practitioners,  much  more  so,  for 

f example,  than  the  traditional  curriculum  consultants  who  went  out 

j from  the  states.  They  were  appreciated  probably  because  they  didn't 

[ have  an  axe  to  grind.  They  provided  information,  and  also  they 

j’  were  trained  in  such  a mode  that  they  went  to  practitioners  and 

listened  to  their  problems.  They  stayed  around,  and  they  came  back, 
b and  they  did  things  that  consultants  often  don't  do.  They  tried 

to  apply  in  working  with  practitioners  what  we  really  knew  was  good 
process  for  research  ut i 1 i zat i on--that  indeed  many  of  the  user 
^ people  here  this  morning  talked  about.  It  turned  out  to  be  just 

I a darned  good  idea.  But  the  results  of  the  research  study  were 

not  utilized,  because  of  course  things  change  in  Washington  and 
[ one  of  the  things  they  did  find  out  was  that  it  was  going  to  cost 

j some  money.  But  NIE  has  maintained  some  semblance  of  a program 

' for  installing  linkers  in  state  departments  of  education.  They 

I are  trying  to  encourage  the  states  to  build  their  own  capacities 

I and  are  giving  the  states  and  other  agencies  subsidies  for  short 

1 periods  of  time  so  they  can  build  in  the  kind  of  linking  agents. 


with  the  supporting  information  services,  that  would  be  required  to 
make  this  thing  work.  They  are  moving  currently  and  I think  are  one 
of  the  reasons  why  I'm  pleased  to  be  working  with  NIE  as  a consultant. 
(Actually  I'm  what  they  call  a Visiting  Fellow;  I'm  there  for  a year 
or  so  and  I have  fairly  wide  latitude  in  the  kinds  of  things  I can 
look  into.) 

NIE  is  also  trying  to  establish  a nationwide  system  of  regional  resource 
centers — not  unlike  the  kinds  of  things  that  Judah  and  Bert  mentioned 
--to  act  as  a feeder  mechanism,  something  like  the  role  of  the 
extension  specialist.  They  are  trying  to  transform  a set  of  insti- 
tutions called  Regional  Educational  Laboratories,  of  which  there 
were  at  one  time  20  and  about  12  are  left  now.  They  are  trying,  in 
effect,  to  knit  together  an  institutional  mechanism  that  will  be  an 
effective  vehicle  for  research  utilization. 

This,  roughly  speaking,  is  what  NIE  is  up  to  in  RU,  and  I commend 
it  to  you,  along  with  the  other  models  that  were  discussed  here,  in 
terms  of  thinking  about  an  institutional  mechanism  for  research 
utilization,  something  that  for  one  thing,  has  a memory  for  past 
research.  I would  also  like  to  note  that  you  can  think  of  the 
function  of  research  utilization  in  two  ways,  one  of  which  is  to 
add  it  to  existing  responsibilities,  sort  of  as  extra  duty.  For 
example,  yesterday  it  was  suggested  that  the  role  of  research  manager 
(which  is  damn  well  a full-time  job  already)  should  also  include 
the  function  of  research  utilization  specialist,  the  person  who 
monitors  the  whole  research  utilization  activity.  And  there  have 
been  suggestions  by  researchers  that  users  have  a responsibility 
for  the  pul  1 of  research  utilization  and  on  the  other  hand  sugges- 
tions by  users  that  researchers  have  a respons i bi 1 i ty  for  the  push 
of  research  utilization.  These  are  all  ideas  about  research  utili- 
zation as  a function  that  is  added  onto  what  people  are  already 
doing.  Now,  the  significance  of  thinking  about  these  institutional 
ideas  is  that  we  are  pointing  out  some  other  options  that  are 
available  and  that  I don't  believe  have  been  seriously  considered 
by  the  military.  In  particular,  we  are  suggesting  that  some  of 
these  research  utilization  jobs  are  full-time  jobs.  Then,  if  we 
come  to  the  point  where  we  realize  that  something  is  important  we're 
eventually  going  to  have  to  start  thinking  not  only  about  getting 
some  money  to  do  it,  which  seems  to  be  a big  problem  (that  is, 
nobody  can  quite  figure  out  whether  it's  6.1,  6.2,  etc.)  but  also 
about  getting  trained  people  to  do  it. 

I won't  take  the  time  now  but  I was  going  to  tell  you  how  we  had 
tried  to  look  across  several  federal  agencies  to  see  whether  there 
was  a common  pattern  in  research  utilization  activities.  Thanks  to 
some  support  from  Judah  Drob's  office  we  were  able  to  look  at  what 
was  happening  in  the  various  agencies--in  Howie  Davis's  agency,  in 
what  was  then  the  Educational  Communications  Institute,  and  in  the 
rehabilitation  field--and  what  we  came  up  with  was  a framework 
within  which  we  could  look  at  research  utilization  functions.  The 
basic  model  was  that  you  had  to  have  a dialogue  between  the  research 
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community  on  the  one  hand  and  user  communities  on  the  other.  And  j 

there  were  certain  functions  that  had  to  take  place  to  make  that  ' 

dialogue  work,  both  commun icat ion-of-resul ts  functions  and  communi-  i 

cat ions-of-needs  functions  that  had  to  interconnect  the  two  ■ 

communities  on  both  the  micro  level  of  person-to-person  communication 
and  on  the  much  larger  and  long-term  level.  We  did  produce  a 
analytical  report  which  covered  research  utilization  across  the 
agencies.  So  I think  that  if  you're  really  interested  in  the 
question  of  institutionalizing  and  also  looking  at  the  gaps  in 
military  research  utilization  for  each  of  the  services,  this  is  one 
kind  of  alternative  to  follow. 

The  last  thing  I'd  like  to  leave  with  you  if  you  are  interested  in 
research  utilization  is  that  in  the  literature  there  is  a growing 
accumulation  of  research  information  on  research  utilization.  I 
would  suggest  that  you  start  thumbing  through  the  volume  Putt i ng 
Knowledge  to  Use  if  you  have  not  already  and  at  least  use  it  as  a 
resource  when  you're  thinking  about  this  in  the  future.  It  is 
probably  the  best  and  most  current  key  to  what's  available  in  the 
literature  on  research  utilization.  I suppose  if  you're  interested 
in  research  utilization  it's  logical  that  you  would  utilize  the 
research  on  the  subject.  It's  logical,  I said. 

Finally,  nobody  has  mentioned  Glenn  Bryan's  paper,  that  was  cir- 
culated before  the  conference.  I thought  it  was  a pretty  good  paper. 

I sometimes  use  other  people's  stuff  myself,  not  often.  I think 
the  concept  of  symbiosis  that  Glenn  used  is  particularly  relevant, 
and  I would  like  to  call  that  to  your  attention  and  ask  you  to  take 
another  look  at  his  paper  because  in  a rather  informal  and  I think 
very  readable  way,  he  really  has  presented  the  main  issues  of 
research  utilization  for  our  consideration.  I remind  you  that  he 
mentions  that  the  dictionary  definition  of  symbiosis  is  "any  close 
association  of  two  dissimilar  types  of  organisms."  The  dissimilar 
types  of  organisms  that  we  are  talking  about  here  are,  of  course, 
the  R6D  community  and  what  we  loosely  call  the  community  of  military 
practice.  Glenn  said  that  the  relationship  between  the  two  i 

communities  could  go  one  of  three  ways.  The  first  way  he  labeled  j 

commensa 1 i sm,  where  one  member  of  an  association  is  benefited  while  . 

the  other  is  apparently  unaffected.  1 would  suggest  that  that  I 

probably  is  the  way  a lot  of  people  look  at  research  utilization  when  ‘ 

they're  looking  at  it  pos i t i vely--that  is,  that  somehow  or  other  ! 

practitioners  can  gain  benefits  from  ongoing  research  without  the 

research  community  being  particularly  affected,  and  research  can  < 

continue  on  its  way  in  it's  ivory  tower.  The  second  way  is 
paras i t i sm,  and  I think  that  when  we  get  paranoid  about  this  two 
worlds  phenomenon  that  we  are  dealing  with,  we  do  tend  to  move 
towards  thinking  about  each  other  as  parasites.  In  other  words, 
the  R&D  community  is  quite  capable  of  thinking  of  users  or 
practitioners  or  would-be-users  as  sucking  out  their  vital  fluids, 
and  perhaps  even  more  so  the  practitioners  are  saying,  "Well  look 
at  those  R&D  dollars  going  to  waste.  These  people  clearly  have 
no  interest  in  us;  they're  simply  parasites  on  the  living  body  of 


the  military  machine."  And  indeed,  1 have  been  surprised  at  the 
level  of  hostility  that  seems  to  exist  in  this  area  in  the  military 
field  when  I can't  think  of  any  field  in  the  behavioral  sciences 
where  there  are  more  positive  examples  of  good  research  utilization. 

But  be  that  as  it  may,  I would  suggest  that  the  true  state  of 
affairs  ought  to  be,  can  be,  and  when  it's  really  working  at  places 
like  HumRRO,  is  mutua 1 i sm,  where  both  members  are  benefited  by  their 
living  together.  Thanks,  and  I'll  take  whatever  questions  you  may  have. 

QUESTION;  Are  copies  of  the  blue  book  available? 

DR.  HAVELOCK;  You  should  have  this,  Putting  Knowledge  to  Use;  A 
Distillation  of  the  Literature  Regarding  Knowledge  Transfer  and  Change. 

1 think  if  you  need  other  copies  the  sponsor  (Howard  Davis)  and  the 
doer  (Ed  Glaser)  are  here.  The  author  is  the  Human  Interaction 
Research  Institute. 

QUESTION;  Is  the  comparative  study  that  you  mentioned  available? 

DR.  HAVELOCK;  It  is  called  RsD  Ut i 1 i zat ion  S t ra teg i es  and  Functions: 

An  Analytical  Comparison  of  Four  Systems.  (We  could  learn  a lot 
about  titling  processes  in  research  utilization.)  The  project 
director  was  myself  and  also  David  Lingwood.  Copies  are  available 
from  the  Manpower  Administration,  Judah  Drob's  office.  The  companion 
piece  to  this  was  a report  that  Bert  mentioned.  After  we  had  done 
this  comparative  study  we  also  looked  at  several  RSD  information 
services  including  the  Defense  Documentation  Center  (DDC)  and  the 
NASA  technology  utilization  system,  and  tried  to  compare  them 
according  to  the  same  kind  of  model,  and  we  also  tried  to  conjure 
up  images  of  what  ideal  RSD  utilization  systems  would  be.  This  is 
contained  in  a report  from  the  Rehabilitation  Services  Administration. 
That's  in  NT  I S so  you  can  get  it  out  if  you  want.  It's  also  available 
from  my  institute  at  cost,  which  is  about  eight  dollars  or  something 
of  that  sort. 

Incidentally,  there  might  be  some  general  interest  in  the  process 
we  used  for  coming  up  with  the  four  alternative  models  of  research 
utilization  systems.  The  idea  was  suggested  to  me  by  Ed  Glaser, 
who  had  done  a similar  kind  of  study  previously.  V/hat  I did  was 
get  together  three  teams  of  experts  on  research  utilization,  but 
the  three  teams  represented  somewhat  different  points  of  view 
towards  the  RU  process;  their  writings  reflected  a different  orienta- 
tion. Each  one  had  a budget  and  a deadline  to  come  up  with  his  own 
model.  Then  we  got  together  to  see  if  we  could  integrate  a single 
model  that  contained  the  best  elements  of  each.  Again,  if  you're 
really  into  thinking  about  how  to  institutionalize  new  mechanisms 
for  this  sort  of  thing  in  any  of  the  services,  this  might  be  a 
worthy  resource  to  investigate.  The  authors  really  are  not  just 
myself  but  Ed  Glaser  was  the  head  of  one  team  and  in  fact  is  the 
author  of  that  piece,  Ronald  Lippi tt  was  the  head  of  another  of  the 
teams,  and  the  third  one  was  Everett  Rogers  who  is  really  the  world's 
foremost  authority  on  the  diffusion  of  innovations.  Thank  you. 
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OVERALL  RESEARCH  UTILIZATION  PERSPECTIVE 


SUMMARY 


Dr,  Thomas  Baaker^  Human  Interaction  Research  Institute,  and  Dr. 
William  McClelland,  HumRRO,  identified  a number  of  recurring  themes 
in  the  first  day  and  a half  of  the  symposium,  themes  that  are  veri- 
fied in  the  research  literature  on  utilization.  Dr.  Backer  cited 
the  following  common  threads: 

• Research  utilization  takes  a serious  oormitment  of  resources . 

. A multidimensional  RU  activity  in  an  agency  tends  to  work  best 
since  it  enables  you  to  connect  with  different  components  of  the 
potential  user  audience  and  to  have  the  synergistic  effect  of 
several  RU  strategies  operating  conjointly. 

. Personal  contact  is  a key  to  effective  RU. 

. RU  demands  a bona  fide  commitment  of  the  agency  at  the  policy 
level  for  funding,  staff  cooperation,  and  coordination  with  oth- 
er activities. 

, RU  must  be  integrated  with  the  total  R&D  program. 

. Early  involvement  of  RU  staff  in  R&D  programming  is  critical. 

• Technical  assistance  is  extremely  valuable  in  facilitating  RU. 

• Effective  RU  requires  the  willingness  to  take  risk. 

Dr.  McClelland  suggested  these  general  issues: 

. There  are  major  differences  in  goals  among  the  communities  re- 
presented at  the  symposium  (for  example,  as  to  how  the  total 
RU  effort  should  be  apportioned  between  problem-solving  and  de- 
velopment of  tools  to  assist  in  solving  problems) . 

. Similarly,  different  agencies  operate  in  different  contexts 
(with  different  missions,  etc.)  and  adaptation  of  an  R&D  product 
must  take  that  into  consideration. 

. The  structure  of  the  military  enterprise  introduces  RU  problems 
that  are  different  than  in  the  civilian  environment. 

. Although  it  is  convenient  to  refer  to  "the  veer, " there  is  a 
whole  conglomeration  of  users — viith  different  interests. 

. We  need  to  adopt  a broader  perspective  and  seek  solutions  wher- 
ever they  may  exist — inside  or  outside  the  military  establishment. 

. Technology  transfer  is  a specialty  in  and  of  itself. 

. Strategies  for  planned  change  are  legion,  and  selection  from 
among  them  is  a complex  art. 

Dr.  Backer  concluded  this  session  with  a review  of  resources  that 

may  be  particularly  useful  to  people  concerned  with  improving  research 

utilization. 
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OVERALL  RESEARCH  UTILIZATIOM 


COUflOn  THEMES 


Dt.  Thomas  E.  Backer 
Senior  Staff  Associate 
Human  Interaction  Research  Institute 


The  purpose  of  the  few  minutes  that  Bill  McClelland  and  I have  to 
spend  with  you  today  is  to  tie  in  what  some  of  our  previous  five 
speakers  here  have  said  about  research  utilization  and  the  experiences 
they  have  had  in  their  own  agency  settings  and  in  their  own  pro- 
fessional work  with  some  of  the  other  things  that  have  been  said 
at  our  conference  in  the  last  two  days  and  also  with  the  literature 
on  research  utilization.  I'm  going  to  take  a first  crack  at  that 
in  just  a minute. 

As  I look  over  some  of  the  notes  that  I've  been  scribbling  down  here 
this  morning,  I think  that  it  would  probably  be  wise  for  me  not  to 
bore  you  with  quotations  from  the  literature  because  I notice  that 
Howie  Davis  has  quoted  the  Bible  and  Bert  Griffis  has  given  us 
quotes  from  our  founding  fathers.  I'm  afraid  that  quotations  from 
the  research  utilization  literature  might  appear  a little  trivial 
by  comparison,  so  I'm  going  to  get  right  to  what  I feel  are  some 
common  themes  in  what  has  been  said  here  today,  themes  which  are 
verified  in  the  research  literature  on  utilization  and  which  also, 

I think,  express  some  of  the  concerns  that  we've  been  hearing  around 
the  table  and  in  our  small  groups  yesterday  in  this  conference  with 
respect  to  a military  environment.  After  I take  a couple  of  minutes 
to  do  that,  then  Bill  will  comment  on  some  of  the  differences  among 
the  five  agency  perspectives  that  we've  heard  here  today  and  what 
we  might  be  able  to  learn  from  them.  Then  I'd  like  to  take  a couple 
of  minutes  at  the  end  to  continue  a process  that  Ron  Havelock 
started  a few  minutes  ago.  That  is,  I want  to  share  with  you  some 
information  about  resources  in  the  research  utilization  field  and 
how  you  can  get  hold  of  things  that  might  be  of  some  use  to  you  as 
you  address  the  issue  of  research  utilization  in  your  own  settings. 

In  terms  of  the  common  themes  that  I have  heard  from  our  speakers 
today,  and  to  some  extent  from  all  of  the  discussion  at  this  con- 
ference so  far,  I see  the  four  service  delivery  agencies  that  have 
been  described  here--Nat iona 1 Institute  of  Mental  Health,  Department 
of  Labor,  Rehabilitation  Services  Administration,  and  National 
institute  of  Educat ion--as  wrestling  with  basically  the  same 
problem:  how  to  organize  their  R&D  system  for  maximum  payoff.  Indeed, 
that's  really  the  reason  why  we  are  all  here,  to  try  to  get  "more 
bang  for  the  buck"  out  of  research  that  is  done  in  various  organizational 
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settings,  specifically  the  military  at  this  conference.  I think 
the  underlying  motivations  for  this  effort  are  twofold:  First, 
the  increasing  press  for  accountability  of  R&D  systems,  now  mandated 
by  the  Congress  and  by  pressures  from  various  public  interest 
groups;  and  second,  the  awareness  that  the  capacity  of  a service 
delivery  system  to  help  people  in  need,  as  specifically  represented 
by  the  four  agencies  that  have  been  discussed  here  today,  depends 
in  part  on  making  the  best  possible  use  of  knowledge  resources. 

I think  those  two  fundamental  motivations  are  shared  by  all  four 
agencies  that  you  heard  about  today, ^and  as  a result  of  that  their 
research  utilization  efforts  have  a number  of  features  in  common. 

I think  many  of  these  features  are  also  shared  by  research  utilization 
efforts  in  military  organizations  as  well. 

Through  some  judicious  note-taking  here  this  morning,  while  listening 
to  the  gentlemen  that  preceded  me,  I have  identified  eight  such 
common  themes  and  I'd  like  to  tell  you  about  those  now. 

1.  Research  utilization  takes  a real  commitment  of  resources. 

Only  so  much  can  be  done  on  a shoestring.  The  amount  of 
money  now  available  is,  I think,  pathetically  small  in 
comparison  to  the  $20  billion  plus  that  is  invested  in  re- 
search each  year  in  the  kinds  of  environments  that  our  five 
speakers  have  been  talking  about.  As  Howard  Davis  remarked 
in  his  introduction  to  the  blue  book  that  most  of  you  have  a 
copy  of.  Putting  Knowledge  to  Use,  the  knowledge  industry, 
broadly  speaking,  has  paid  remarkably  little  attention  to 
the  utilization  of  its  product,  to  the  marketing  of  its 
products.  In  the  human  service  area  in  particular,  I think 
in  part  this  has  been  because  utilization  efforts  attached  to 
R&D  programs  have  been  underfunded  and  understaffed.  I 
think  we've  heard  some  similar  kinds  of  criticism  of  military 
R&D  organizations  in  the  GAO  report  that  was  discussed  yesterday. 

2.  A multiple  kind  of  research  utilization  activity  in  an  agency 
setting  seems  to  work  the  best.  By  that  I mean  that  the 
research  utilization  programs  that  Howie  and  Bert  and  the 
others  have  described  to  you  today  have  chosen  not  to  put  all 
of  their  eggs  in  one  basket.  And  so  we've  heard  about  the 
support  of  what  you  might  call  basic  research  on  research 
utilization,  we've  heard  about  publication  of  research  summaries, 
about  publication  of  magazines  like  the  two  that  Howie  Davis 
just  mentioned  to  you,  and  we've  heard  about  efforts  to  produce 
improved  (more  readable,  more  usable)  research  reports.  We've 
heard  about  research  utilization  conferences  like  the  one 

we're  all  a part  of  right  now.  We've  heard  about  indexes  to 
the  literature  and  information  systems,  about  consultation, 
about  direct  service  efforts  of  one  sort  or  another--a  whole 
cornucopia  of  activity  in  the  research  utilization  field.  I 
think  that  is  one  theme  that  typifies  the  research  utilization 
efforts  of  all  the  agencies  that  you've  heard  about  today. 

And  I think  it  is  highly  desirable  not  to  concentrate  all 
your  energies  in  one  area,  because  you  can  hit  different  parts 
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of  your  potential  user  audience  at  different  times  if  you  have 
a number  of  things  going  on  rather  than  spending  all  of  the 
money  that  you  have  available  for  utilization  in  one  area. 
Through  that  kind  of  multiple  activity,  in  many  cases  you 
can  achieve  a kind  of  building  or  synergistic  consequence 
in  terms  of  getting  R&D  knowledge  used. 

3.  Personal  activity  and  contact  is  a key  mechanism  for  effecting 
research  utilization.  I think  the  literature  certainly 
supports  this.  In  fact,  if  there  is  one  principle  that  re- 
search on  research  utilization  tells  us,  it  is  that  personal 
contact  is  the  primary  mechanism  by  which  change  takes  place. 
We've  heard  all  of  our  speakers  here  today  say  that  in  differ- 
ent ways.  And  we've  heard  Howie  Davis  talking  about  some  of 
the  very  personal  and  psychological  factors  that  bear  on 
organizational  change  that  he  summarized  under  the  eight 
factor  A VICTORY  model.  So  I think  that  is  another  commonality 
that  characterizes  the  research  utilization  programs  that  we've 
heard  about  today. 

h.  Research  utilization  requires  a real  commitment  of  the  agency 
at  the  policy  level  for  funding,  for  staff  cooperation,  and 
for  coordination  with  other  activities  both  in  the  research 
area  and  outside  of  it.  Another  way  of  expressing  that  is 
what  I've  heard  several  other  people  say  here  today  and  that  is 
that  research  utilization  really  is  everybody's  problem.  I 
think  there  is  a natural  tendency  for  all  of  us  to  do  a little 
bit  of  finger  pointing  and  say,  "Hey,  the  problem  is  really 
over  here  on  this  other  side;  I'm  a researcher;  you  users  are 
the  ones  who  are  all  fouled  up,"  and  for  the  user  to  say  the 
same  thing  from  his  or  her  perspective.  But  I guess  the  way 
I see  it.  and  what  I really  hear  as  a philosophy  that  underlies 
all  of  the  research  utilization  programs  in  these  agency 
settings  that  we've  been  hearing  about  today,  is  that  research 
utilization  does  require  mutual  effort  and  to  the  extent  that 
the  effort  is  one-sided  and  to  the  extent  that  energy  is 
invested  in  finger  pointing  rather  than  sitting  down  and  trying 
to  figure  out  as  a team  what  we  can  do  to  solve  the  common 
problems,  the  research  utilization  effort  is  going  to  function 
less  effectively.  It's  everybody's  problem  and  unless 
everybody  works  together  and  minimizes  the  finger  pointing  and 
thinks  more  in  terms  of  what  we  can  do  to  improve  the  system, 
research  utilization  unfortunately  will  take  a back  seat  to 
political  infighting  and  to  a lot  of  consequences  that  are 
not  very  desirable  in  terms  of  program  improvement. 

5.  Research  utilization  must  be  specifically  tied  in  with  the 

overall  research  and  development  program  in  every  way  possible, 
from  planning  to  evaluation  and  all  the  stages  in  between.  I 
think  we've  heard  Judah  Drob  talk  especially  persuasively  this 
morning  about  the  kinds  of  things  that  can  flow  from  that  sort 
of  close  organization. 
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6.  Early  involvement  of  research  utilization  staff  in  R&D 
programming  is  particularly  critical.  I suppose  that  is 
another  way  of  saying  what  Admiral  Metcalf  said  to  us  yester- 
day, that  planning  is  critical  to  good  research  utilization. 

7.  The  availability  of  technical  assistance  or  consultation  is 
critical.  We've  seen  examples  of  that  today  in  terms  of 
RSA's  experience  with  the  research  utilization  specialist 
and  in  Judah  Drob's  "buddy  system"  that  he  described  for 

us  so  eloquently. 

8.  Research  utilization,  to  be  worthwhile,  requires  the  willing- 
ness to  take  risk,  to  be  wrong,  to  fall  on  your  can.  I 
think  that  is  an  important  point  that  everybody  knows  but 
not  everyone  is  willing  to  acknowledge.  For  those  of  us 

who  come  from  a research  background  and  from  a professional 
discipline  that  stresses  the  purity  of  scientific  inquiry, 
it  is  a little  tough  to  acknowledge  that  you  are  taking  a 
chance  and  that  you  can  be  dead  wrong  about  something.  But 
unless  you  acknowledge  that  to  yourself  as  a researcher  and 
unless  the  system  permits  you  to  make  a mistake  and  not  go 
under  because  of  it,  I think  research  utilization  as  an 
activity  is  going  to  be  far  less  successful,  because  without 
the  ability  to  take  those  risks,  there  are  many  kinds  of 
potential  payoffs  that  you  can  never  get  to. 

So  those  are  the  eight  generalizations  or  themes  that  I see  coming 
out  of  the  discussion  that  we've  had  here  this  morning,  and  to  some 
extent  from  all  of  what  we've  heard  in  the  conference  so  far.  I 
want  to  leave  you  with  one  other  point  that  I think  may  help  in 
conceptualizing  the  issues  that,  hopefully,  we  will  be  bearing  down 
on  a little  bit  more  concretely  in  our  recommendations  session  in 
the  next  couple  of  days.  And  that  is  that  in  your  own  thinking 
about  research  utilization  activity,  it  is  useful  to  think  about 
this  experience  that  we're  all  having  here  together  as  a research 
utilization  activity  itself.  I know  that  may  be  an  obvious  point, 
but  that's  what  it  is,  because  this  conference  has  been  designed 
very  much  along  the  same  lines  as  the  research  utilization  con- 
ference that  Bert  Griffis  mentioned  to  you  that  our  organization 
coordinated  a couple  of  years  ago.  The  objective  of  that  conference, 
in  essence,  was  to  disseminate  the  findings  from  our  evaluation  of 
the  research  utilization  specialist  through  RSA  and  other  funding 
agencies  that  have  supported  the  development  of  the  research 
utilization  conference  as  a strategy  for  effecting  change. 

There  has  been  a literature  accumulation  and  a body  of  practice 
accumulation  that  has  been  followed  and  I think  followed  quite  wel 
in  designing  this  conference.  The  consequence  is  that  if  you 
process  your  own  experience  and  the  kinds  of  reactions  you've 
been  having  as  you  heard  other  people  talk  and  you  reacted  to  the 
activities  of  this  conference,  to  what  we're  trying  to  do,  and 
perhaps  as  importantly,  to  how  we're  trying  to  do  it  and  to  the 
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strategies  that  we've  used  in  attempting  to  effect  a transfer 
of  knowledge,  I think  you  can  get  a good  handle  on  what  research 
utilization  activities  are  like  for  other  populations  of  researchers 
and  practitioners.  You  can  use  your  own  experience  in  that  way  if 
you  process  it  and  you  look  at  it  in  that  fashion. 

That  is  the  final  common  theme  I'd  like  to  leave  with  you  from  the 
discussion  that  we've  heard  today  from  our  five  federal  agency 
perspectives.  I think  each  one  of  them  has  done  a commendable  job 
of  looking  at  their  own  experience  and  processing  it  in  a way 
that  is  relevant  to  the  needs  of  this  conference,  thereby  making 
that  experience  useful.  Each  one  of  you,  in  terms  of  the  things 
that  have  been  happening  to  you  over  the  period  of  the  last  two 
days  and  what  will  be  happening  in  the  next  couple  of  <jays,  probably 
will  come  away  from  this  conference  with  some  new  knowledge  yourselves 
about  how  to  do  research  utilization  and  what  works  and  what  doesn't. 
It's  there  for  the  taking,  and  so  my  invitation  to  you  all  is  to 
give  it  a try  and  see  if  you  can  use  it. 

That's  all  I have  to  say  right  now;  are  there  any  questions  that 
anyone  would  like  to  raise  right  at  this  moment?  If  not,  let's 
go  on  to  Bill  McClelland. 
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The  listings  that  we  have  heard  from  the  speakers  who  have  stood 
before  you  have  frequently  added  up  to  the  magic  number  seven,  plus 
or  minus  two,  and  it  is  indeed  with  some  effort  that  I have  followed 
established  HumRRO  tradition  to  come  up  with  seven  points  that  I 
would  like  to  make.  They  are  on  general  thematic  issues  which  I 
have  read  in  the  materials  for  this  conference  or  have  extracted 
from  what  has  been  said  by  speakers  before  us  this  morning  and  by 
those  with  whom  I have  attended  group  sessions.  I want  to  say 
just  a sentence  or  two  about  each  of  the  seven  issues:  goals, 
contexts,  structure,  roles,  what  is  to  be  utilized,  technology 
transfer,  and  change  strategies.  Since  there  are  people  here  who 
have  written  books  on  one  or  more  of  these  individual  points,  ob- 
viously I'll  be  very  brief. 

First,  in  the  matter  of  goals,  there  are  clearly  some  major 
differences  among  those  of  us  who  are  here.  It's  quite  clear  that 
the  emphasis  of  users,  of  sponsors  and  of  operators  has  been  very 
strong  on  problem  solving,  and  I think  that's  an  emphasis  that  most 
of  us  can  accept  for  at  least  part  of  the  activity  in  which  we  are 
engaged  in  human  RDT&E  efforts.  However,  there  are  some  fairly 
major  differences  among  us  as  to  how  much  of  the  total  effort  should 


Dr.  William  A.  McClelland 
President 

Human  Resources  Research  Organization 


1 


1 

1 


-107- 


be  devoted  to  the  equally  desirable,  if  not  essential,  goal  of 
developing  tools  in  order  to  solve  problems.  This  issue  keeps 
coming  up  in  terms  of  whether  we're  talking  about  research  or 
development.  I note  two  things  being  stressed  in  the  papers,  talks 
and  discussion.  I hear  two  emphases.  One  is  on  research  and  one  is 
on  what  is  called  research  but  what  is  really  developmental  activity. 

I think  that's  a distinction  we're  going  to  have  to  be  sure  to  keep 
fairly  firmly  in  mind.  Research  is  not  development.  It  is  not 
technical  advisory  service.  They  are  different  ballparks. 

Second,  as  to  context,  1 hear  some  differences  here  among  us  and 
with  the  agencies  involved,  both  in  terms  of  the  mission  of  that 
particular  agency  and  in  terms  of  the  time  or  the  climate  in  which 
we  happen  to  be  operating.  Meredith  gave  a very  lucid  statement, 
which  is  consistent  with  my  experience  at  HumRRO,  in  which  he  talked 
about  a kind  of  organization  which  existed  at  a particular  time. 

The  parallel  to  that  now  clearly  has  to  be,  it  seems  to  me,  the 
in-house  laboratories.  The  contexts,  1 think,  are  different  in  a 
variety  of  ways.  I'm  not  going  to  belabor  that  particular  point, 

I'm  simply  saying  that  in  the  research  utilization  process  it's 
important  to  realize  that  adaptations  that  are  made  have  to  take 
these  sorts  of  variables  into  account. 

Third,  I want  to  say  a word  about  the  structure  of  the  organization. 
When  I speak  of  structure  I'm  including  the  trappings  that  come 
from  the  regulations  and  the  rules  and  the  procedures  which  in  many 
cases  are  quite  laboriously  developed  in  order  to  make  up  a system 
that  will  enhance  the  likelihood  of  utilization  of  R£0.  A lot  of 
different  structures  make  sense  here.  I'm  a little  frightened  at  times 
(and  this  stems  directly  from  the  kind  of  experience  that  I have  had) 
of  the  complexity  of  interpersonal  relationships  at  different 
hierarchical  levels  when  we  are  dealing,  as  we  are,  with  at  least 
three  or  four  different  and  definable  organizational  levels  within 
the  Department  of  Defense.  I don't  say  the  situation  is  any  simpler 
in  the  civilian  settings  about  which  previous  speakers  have  talked, 
but  certainly  hierarchically  it's  somewhat  different.  I don't  worry 
too  much  about  the  differences  between  the  civilian  and  the  military 
contexts  on  this  point  of  structure.  I am  simply  pointing  out  that 
we  face  different  problems  in  these  two  environments.  I don't  think 
that  necessarily  invalidates  some  of  the  inferences  that  can  be 
drawn  from  what  has  been  said  earlier. 


A fourth  point  has  to  do  with  roles.  I think  we're  talking  about 
different  roles.  In  fact  we've  had  a fourth  one  introduced  this 
morning  when  we  talked  about  the  user  and  the  researcher  and  the 
research  manager  and  added  to  that  the  research  utilization  special 
These  are  indeed  different  roles.  I'd  also  like  to  point  out, 
however,  that  there  is  no  such  thing  as  "the  user."  I'm  a little 
disturbed  by  some  of  the  comments  I have  heard  which  tend  to  make 
unequivocal  distinctions  among  people  operating  at  various  levels. 


Admittedly  there's  a priority  here,  but  I do 
lot  of  different  hierarchies  are  involved,  a 
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are  involved,  and  that  their  interests  obviously  have  to  be  considered 
before  a commitment  is  made  to  undertake  a project  which  hopefully 
will  solve  the  particular  operational  problem.  So  we  aren't  talking 
about  individual  users,  we're  talking  about  a family  of  users  and  I 
think  their  interests  are  different  and  have  to  be  somehow  taken  into 
account.  I think  that's  true,  incidentally,  of  civilian  contexts  as 
well,  and  the  parallel  there  is  probably  a lot  closer  than  one  might 
suspect  at  first  thought. 

Fifth,  I'd  like  to  say  a bit  about  the  event  itself.  I'd  like  to 
applaud  Ron  Havelock's  emphasis,  which  Meredith  also  mentioned, 
on  what  is  to  be  utilized,  what  is  to  be  put  into  practice,  what 
is  to  contribute  to  change.  It's  not  Just  what  is  currently  being 
done.  It  should  include  that  which  has  been  done  by  your  agency, 
by  related  service  agencies,  and  should  also  consider  what  has  been 
done  outside  the  military  context  which  might  have  relevance  to  a 
particular  presenting  problem.  The  "not  invented  here"  phenomenon 
is  a very  insidious  one  and  it's  one  that  frequently  we  will  find 
in  our  own  labs  and  our  own  organization,  such  as  people  being 
unwilling  to  cite  the  work  of  someone  working  next  door  to  them. 

As  far  as  research  utilization  is  concerned,  I think  that's  a 
completely  unforgivable  perspective.  We  talked  about  the  "here  and 
now"  phenomenon.  I think  our  perspective  has  to  be  broader.  And 
to  restate  what  Glenn  Bryan  and  Jay  Uhlaner  and  others  have  said 
earlier,  I think  our  expectations  for  utilization  must  vary  as  a 
function  of  the  different  kinds  of  events.  The  same  requirements 
cannot  be  posed  for  each. 

Sixth,  technology  transfer,  I think  you  will  now  agree,  is  a specialty 
in  and  of  itself.  Most  of  us  are  practitioners  of  this  particular 
process  but  we  are  not  experts  in  it.  There  are  a few  living 
experts,  but  for  the  most  part  they  are  not  here  today.  It  is  a 
specialty  area  and  if  we're  to  take  seriously  the  kinds  of  demands 
that  are  being  made  upon  the  R&D  community  we'd  better  recognize  that 
fact  and  recognize  it  more  fully  than  we  have  done  in  the  past. 

Tom  has  already  spoken  to  that  particular  point. 

Finally,  change  strategies  are  many.  The  change  strategies  that 
are  going  to  vrark  are  varied  and  the  application  of  these  strategies 
is  an  art.  The  choice  of  change  strategy  is  going  to  be  a function 
of  a great  number  of  things,  one  of  which,  as  has  been  suggested, 
is  the  category  of  RsD  that's  involved.  I would  simply  call  your 
attention  to  all  these  other  points  that  I've  tried  to  mention  as 
points  of  difference,  that  is:  goals,  context,  organizational 
structure,  roles,  and  the  events.  All  can  influence  the  kind  of 
strategy  which  might  be  most  effective.  It  is  an  art,  we  need 
specialists  in  its  practice,  and  we  need  to  learn  more  about  it 
ourse 1 ves , 

RESOURCES  FOR  RESEARCH  UTILIZATIOri 

DR.  BACKER:  There  are  a few  things  that  I'd  like  to  mention  in 


terms  of  resources  for  research  utilization.  We've  already  heard 
about  one  that  is  quite  important,  the  blue  book  that  most  of  you 
have  a copy  of.  Putting  Knowledge  to  Use,  and  access  channels  have 
been  arranged  so  that  you  can  get  a copy  free  if  you  don't  already 
have  one.  That  is  the  most  recent  review  of  the  literature,  However, 
it  is  not  the  only  review.  t think  that  it  is  worth  mentioning 
two  other  important  reviews  of  the  literature  on  research  utilization 
and  organizational  change.  Both  of  them  are  referenced  in  the  blue 
book.  So  if  you  have  a copy  of  the  blue  book,  you  can  also  get 
the  citation  data  for  the  other  two  that  I'm  going  to  mention  to 
you . 

One  of  them  was  published  in  1969  and  was  authored  by  Dr.  Havelock, 
and  the  title  is  Planning  for  Innovation.  At  the  time  it  came  out, 
it  was  certainly  the  most  comprehensive  and  up-to-date  review  of 
what  had  been  done  to  that  point  in  the  knowledge  utilization  field. 

It  includes  quite  an  extensive  bibliography  as  well  as  Ron's  analysis 
of  the  whole  problem  of  research  utilization  from  a systems  per- 
spective. That  is  one  resource  that  I commend  to  all  of  you  in 
order  to  get  into  the  literature  in  nK>re  detail  if  you  have  not  had 
that  opportunity  or  responsibility  already. 

The  third  volume  of  the  three  volume  bible  on  research  utilization 
is  Everett  Roger's  book,  Com.mun  i cat  ion  of  Innovations,  published  in 
1971.  That  again  contains  a bibliography.  It  also  includes 
Dr.  Roger's  analysis  of  the  question  of  how  innovations  get  diffused. 
His  particular  theme  is  the  diffusion  of  innovations  in  fairly  large 
organizations  or  throughout  a society.  He  offers  a rather  inter- 
esting set  of  some  220  generalizations  about  what  promotes  effective 
diffusion  and  use  of  knowledge  in  a social  setting.  That  is  something 
not  duplicated  anywhere  else  in  the  literature.  I think  it  would  be 
worthwhile  for  anyone  who  is  a serious  student  of  this  subject  to 
get  acquainted  with  that  list  of  generalizations,  because  some  of 
them  are  immediately  relevant  to  real  life  problems  you  may  be  facing. 

I would  like  to  mention  a couple  of  other  general  resources  which 
are  also  referenced  in  the  blue  book,  so  that  you  can  get  the 
citation  data  if  you  want  it.  One  is  the  Directory  of  Federal 
Technology  Transfer.  The  most  recent  edition,  I believe,  was  in 
1 976 . Th i s book  was  developed  by  the  National  Science  Foundation 
under  the  auspices  of  the  Federal  Council  for  Science  and  Technology. 
It  provides  capsule  descriptions  of  a great  number  of  technology 
transfer  and  research  utilization  programs  in,  I believe,  50  or 
60  federal  agencies.  Some  of  the  agency  settings  or  organizations 
from  the  military  that  are  represented  in  this  room  have  probably  been 
described  in  that  book.  All  of  the  programs  we've  heard  about  from 
our  speakers  here  this  morning  are  also  described  in  the  book.  And 
I think  you  might  call  it  sort  of  a "Whole  Earth  Catalog"  of  research 
utilization  and  technology  transfer.  As  such,  it  is  a convenient  and 
easy  way  to  get  an  idea  of  what  people  are  doing  in  this  area, 
because  in  four  or  five  pages  of  material  on  each  program  you  get  an 
overview  of  what  is  happening,  what  kinds  of  activities  are  being 
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pursued,  what  sorts  of  publications  are  being  put  out,  and  what  anrount 
of  money  is  available  in  different  programs  to  support  research 
utilization  and  technology  transfer,  I guess  if  I had  to  recommend 
one  reference  for  learning  what's  happening  in  this  specialized 
field  that  Bill  just  made  reference  to,  that  would  be  it.  I think 
it  is  a very  useful  way  of  getting  started  in  understanding  a very 
complex  and  not  terribly  well  interconnected  set  of  activities  in 
different  public  settings  which  affect  the  use  of  new  knowledge  in 
practitioner  environments. 

So  that  is  another  resource.  What  you  might  refer  to  as  the  "Yellow 
Pages"  of  technology  transfer  and  research  utilization  is  a publi- 
cation entitled  Technology  Transfer  Directory  of  People.  The  most 
recent  edition  came  out in  1976  and  i understand  a new  edition  of 
that  publication  is  underway.  In  any  event,  the  Technology  Transfer 
Pi  rectory , Just  like  a telephone  book,  gives  names  and  addresses 
of  people  who  are  active  in  the  research  utilization  and  technology 
transfer  field.  I expect  that  a fair  number  of  people  here  in  this 
room  are  in  that  directory.  It  is  a good  place  to  get  names  of 
people  you  might  want  to  contact  with  reference  to  a particular 
problem  in  the  research  utilization  area. 

I would  also  like  to  reiterate  what  Bert  Griffis  very  modestly 
mentioned  a little  bit  earlier  today-the  Rehabilitation  Services 
Agency  has  prepared  their  guidelines  for  research  utilization  and 
for  preparing  R£D  reports  in  a fashion  that  will  promote  their 
utilization.  I think  they  are  very  excellent  publications  that 
might  be  worth  getting  a copy  of  and  reproducing  and  distributing 
to  everyone  in  your  own  organization  who  has  a responsibility  for 
generating  research  knowledge.  This  set  of  publications,  in  a 
short  number  of  pages  and  a small  number  of  words,  gives  a lot  of 
practical  guidance  about  how  to  write  things  so  that  people  will 
use  them  and  about  how  to  do  research  so  that  the  results  will  be 
more  easily  utilizable.  Both  publications  are  also  mentioned  in 
the  blue  book,  so  you  can  get  the  citation  data  from  there. 

Another  publication,  kind  of  an  appendix  to  Putting  Knowledge  to  Use, 
is  this  little  booklet  called  Information  Sources  and  How  to  Use  Them. 
It  should  be  possible  to  get  a copy  of  it  either  through  us  or  through 
Howie  Davis.  If  you  leave  your  name  with  us  here  today,  we'll  be 
happy  to  send  you  a copy  free  of  charge.  Information  Sources  is  a 
min i di rectory  to  the  different  kinds  of  information  systems  and 
publications  that  are  available  in  a broad  variety  of  areas  and  that 
give  people  access  to  research  information  or  information  from 
different  fields  of  practice.  It  includes  a description  of  the 
various  computerized  information  retrieval  systems,  including  ERIC 
and  NTIS,  which  Ron  Havelock  mentioned,  and  a great  variety  of 
others.  If  you  want  to  know  how  to  get  information  in  almost  any 
subject  area,  this  is  a good  place  to  start.  It  also  includes 
the  names  and  addresses  and  brief  descriptions  of  a number  of 
publications,  indexes,  periodicals,  and  various  other  sources  that 
may  be  helpful  to  you  if  you  want  to  get  information  on  any  given 


subject.  I think  it  is  a handy  reference  guide  you  might  want  to 
have  on  your  desk  in  case  you  need  to  get  Information  about  almost 
anything.  It  has  that  kind  of  broad  utility  and  relevance  to  a lot 
of  different  subject  areas. 

Those  are  the  resources  that  I have  to  talk  about  in  terms  of  print 
publications.  I'd  like  to  wrap  up  this  part  of  our  discussion  session 
by  reminding  you  again  of  a very  obvious  point  I'm  sure  you've  thought 
of,  that  print  resources,  as  valuable  as  they  are,  are  only  the 
beginning  of  research  utilization  or  knowledge  transfer  or  or- 
ganizational change,  all  of  the  things  we've  been  talking  about  here. 
Once  again,  personal  contact  is  the  way  most  organizational  changes 
take  place.  Although  you  can  get  information  out  of  this  booklet 
or  the  blue  book  or  the  other  books  that  I've  mentioned  to  you 
another  very  valuable  resource  that  is  accessed  in  part,  but  only 
in  part,  by  publications  like  this  are  people.  People  you  run  into 
contact  with  in  this  conference  and  people  you  might  be  able  to 
contact  through  the  mechanism  of  the  Technology  Transfer  Directory 
of  People  can  be  equally,  or  in  many  cases  more,  useful  to  you  than 
anything  that  you  can  get  in  print,  because  you  can  have  a two-way 
communication  with  them.  The  inherent  limitation  of  any  book  is 
that  although  you  can  read  it,  it  can't  talk  back  to  you.  It  can 
only  give  you  information  to  the  extent  that  it  is  already  worked 
into  the  pages  and  happens  to  be  relevant  to  your  needs,  whereas 
with  the  dialogue  that  is  made  possible  by  two  people  sitting  down 
and  talking,  there  can  be  some  tailoring  of  the  information  and 
therefore  a lot  more  specificity  in  terms  of  whatever  your  needs 
might  be. 

So,  using  the  di rectory  that  I mentioned  to  you  a few  minutes  ago 
and  contact  that  may  have  been  established  through  this  meeting  or 
others  are  other  ways  to  get  information  you  may  need  about 
research  utilization.  Thank  you. 
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TECHNOLOGY  TRANSFER  PANEL  DISCUSSION 


SUMMARY 


The  principal  focus  of  this  session  was  on  the  transfer  of  technology 
from  the  Federal  laboratories  to  state  and  city  jurisdictions. 

Allan  Sjoholm,  San  Diego  Technology  Action  Center,  defined  the  crit- 
ical need  for  new  technology  to  reduce  costs  and  improve  operating 
efficiency  in  local  governments. 

George  Linsteadt,  Federal  Laboratory  Consortium  for  Technology  Transfer, 
described  the  role  of  the  Consortium  in  facilitating  secondary  research 
utilization  or  technology  transfer.  He  also  recounted  the  historical 
evolution  of  government  policy  relative  to  technology  transfer.  Fi- 
nally, he  discussed  a number  of  obstacles  to  the  transfer  of  Federal 
laboratory  technology,  including  budget  limitations;  statutory  restric- 
tions (such  as  the  Mansfield  Amendment,  which  precludes  the  use  of 
defense  R&D  for  applications  that  do  not  have  a direct  relationship 
to  the  DoD  mission);  lack  of  policy  in  Federal  agencies  supporting 
technology  transfer;  lack  of  information  in  local  settings  regarding 
laboratory  capabilities;  poor  communication  between  federal  agencies 
and  local  governments;  lack  of  technical  assistance  to  support  tech- 
nology adaptation;  and  the  problem  of  competition  with  private  industry. 

Gerald  Miller,  State  of  Oregon,  cited  a series  of  concrete  examples  of 
instances  where  he,  in  the  capacity  of  Technology  Transfer  Coordinator, 
has  been  able  to  track  down  technology  produced  by  the  Federal  labora- 
tories and  bring  it  to  bear  to  solve  critical  problems  faced  by  cities, 
counties,  and  states.  He  urged  the  conference  to  get  involved  in  this 
process . 

Charles  Miller,  Lawrence  Livermore  Laboratory,  described  several  ac- 
tivities of  that  Laboratory  in  support  of  technology  transfer.  He 
also  reported  a number  of  conclusions  regarding  successful  transfer 
activities : 

. The  activity  must  be  a full-time  directed  effort  by  the  technology 
source . 

. The  receptors  in  the  local  government  must  be  active  and  informed. 

. The  technology  transfer  agent  must  have  access  to  a broad  spectrum 
of  technical  information,  and  must  have  freedom  to  act  and  drive 
to  respond  to  needs. 

. Personal  contact  is  crucial. 

• Disseminating  reports  is  not  technology  transfer. 

• Transfer  is  an  "integrating"  process. 

D.  M.  DelaBarre,  California  Innovation  Group  (CIG)  related  the  evolution 
of  CIG  as  an  important,  institutionalized  mechanism  for  putting  together 
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a team  from  the  public  and  private  sectors  to  assist  in  technology 
transfer  to  local  governments. 

John  Lockwood,  City  of  San  Diego,  discussed  the  National  Consortium 
for  Technology  Initiatives , a group  of  large  cities  and  urban  counties 
that  have  formed  an  alliance  for  the  purpose  of  identifying , priori- 
tizing and  addressing  corrmon  technological  needs. 

Robert  Crawford,  National  Science  Foundation,  described  the  activities 
of  the  Intergovernmental  Programs,  a component  of  the  RANN  program 
of  NSF,  as  another  vehicle  for  supporting  the  technological  needs  of 
state  and  local  government. 

Finally,  Dr.  Kay  Inaba,  XIZYX  Information  Corporation,  endeavored  to 
summarize  the  technology  transfer  session  from  a practitioner's  point 
of  view.  He  urged  that: 

. Researchers  stick  to  research  and  leave  applications  to  practi- 
tioners. 

. More  research  be  conducted  on  the  applications  process, 

. More  attention  be  devoted  to  procurement  practices,  the  systems 
approach,  and  the  optimal  timing  of  moving  projects  from  research 
to  application. 

. The  effectiveness  of  the  consortium  concept  be  recognized  and 
further  exploited. 

. A truce  be  declared  between  researchers  and  practitioners,  and 
that  they  collaborate  in  the  development  of  a series  of  appli- 
cations program  plans. 
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TECHNOLOGY  TRANSFER  PANEL  DISCUSSION 


DR.  GLASER;  I just  want  to  take  about  three  minutes  to  read  a rather 
important  statement  by  Frank  Press,  Director  of  the  Office  of  Science 
and  Technology  Policy.  Executive  Office  of  the  President,  and  George 
Busbee,  Governor  of  Georgia.  The  statement  appeared  as  an  editorial 
in  Science  Magazine,  27  May.  They  say: 

There  is  a growing  awareness,  both  in  Washington  and  through- 
out the  country,  that  if  science  and  technology  are  to  benefit 
our  people  more  effectively,  a better  R&D  partnership  must  be 
established  between  the  federal  government  and  the  states, 
counties  and  cities.  Properly  designed  and  directed  towards 
state  and  local  needs,  federally  supported  R&D  could  help  to 
protect  regional  and  local  environments,  reduce  demands  on 
energy  and  various  natural  resources,  and  improve  the  delivery 
of  state  and  local  services. 

In  recent  years,  federal  funding  of  R&D  for  the  civilian 
sector  has  been  growing  rapidly.  It  is  now  in  excess  of  $7 
billion  annually.  But  its  impact  on  meeting  public  expecta- 
tions--on  filling  the  everyday  needs  of  the  people--often 
seems  disappointing. 

Past  intergovernmental  science  and  technology  programs  have 
been  primarily  one-sided  affairs,  relying  heavily  on  the 
federal  government  offering  money  and  off-the-shelf  technology 
to  the  states  and  cities  with  little  concern  for  or  under- 
standing of  the  user's  needs.  Much  of  this  technology  was  a 
spin-off  of  aerospace  and  military  R&D  which  might  be  adaptable 
to  the  needs  of  a local  fire  or  police  department  or  serve 
some  other  public  need.  In  addition  there  were  programs  to 
supply  governors  and  mayors  with  science  and  technology  advisors. 

These  efforts  at  domestic  technology  transfer  in  several  cases 
have  met  with  some  success.  But  the  residual  problems  of  the 
state  and  local  levels  remain  enormous  and  deserving  of  a 
greater  effort.  Clearly  some  new  stimuli  and  new  approaches 
are  needed. 

And  that's  what  A1  Sjoholm  and  his  colleagues  will  be  discussing  this 
afternoon . 
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INTRODUCTION 


Mv.  Allan  A.  S.joholm 

Director,  San  Diego  Technology  Action  Center 
and  Navy  Personnel  R&D  Center 


For  the  next  two  hours  we  would  like  to  change  the  scene  just  a little 
bit  and  talk  about  technology  transfer  with  primary  emphasis  upon 
local  government--an  area  close  to  all  of  us. 

Essentially,  this  involves  the  movement  and  utilization  of  such 
things  as  existing  procedures,  techniques,  and  equipment  from  the 
Federal  laboratories  and  other  sources  into  scate,  county,  and  city 
J ur i sd i ct i ons . 

Today's  local  governments  are  faced  with  budget  limitations,  resource 
constraints,  rising  costs  and,  at  the  same  time,  demands  for  additional 
services.  They  cannot  realistically  afford  a large  research  and 
development  investment  and  yet  urgently  need  new  technological 
developments  to  reduce  costs  and  improve  operational  efficiency. 

How  does  this  relate  to  Federal  laboratories?  To  quote  Mayor  Kenneth 
A.  Gibson  of  Newark,  New  Jersey,  "The  $26  billion  in  national  invest- 
ment in  science  and  technology  has  not  yet  begun  to  be  significantly 
tapped  in  terms  of  its  potential  to  benefit  state  and  local  govern- 
ments. The  Federal  laboratories  represent  a $12  billion  publicly 
owned  gold  mine  of  national  knowledge,  methodologies,  and  end- 
products  which  could  be  devoted  to  local  government  service."' 

There  is  ample  expressed  concern  and  emphasis  from  the  White  House 
level.  To  quote  former  President  Gerald  Ford: 

...the  progress  we  seek  requires  a new  partnership  in 
science  and  technology--one  which  brings  together  the 
Federal  government,  private  enterprise  and  state  and 
local  governments,  and  our  universities  and  research 
centers  in  a coordinated,  cooperative  effort  to  serve 
the  national  interest....  Only  if  this  happens .. .can 
we  be  sure  that  our  scientific  and  technological  re- 
sources v/i  1 1 be  used  as  effectively  as  possible  in 
meeting  our  priority  national  needs. 2 

President  Carter  stated  his  perspective  in  a communique  to  his  cabinet 
agencies  instructing  them  to  seek  local  government  involvement  because: 

. State  and  local  sectors  constitute  the  delivery  mechanisms 
for  most  of  the  actual  services  the  Federal  government 
provides . 
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State  and  local  concerns,  as  well  as  their  expertise,  should 
be  considered  as  programs  are  being  developed  in  order  to 
ensure  the  practicality  and  effectiveness  of  the  programs. 

Such  early  pai t i c i pat  ion  by  state  and  local  officials  in  our 
planning  process  will  help  ensure  broad-based  support  for  the 
proposals  that  are  eventually  developed. 

. It  will  ensure  that  priorities  developed  at  the  Federal  level 
will  work  in  conjunction  with,  and  not  at  cross  purposes  to, 
priorities  at  the  state  and  local  level. ^ 

And  one  final  quotation,  a very  important  one  from  the  May  27,  1977, 
editorial  in  Science  Magazine  entitled.  "Intergovernmental  Science 
and  Technology": 

There  is  a growing  awareness,  both  in  Washington  and 
throughout  the  country,  that  if  science  and  technology 
are  to  benefit  our  people  more  effectively,  a better 
R&D  partnership  must  be  established  between  the  federal 
government  and  the  states,  counties,  and  cities. 

Properly  designed  and  directed  toward  state  and  local 
needs,  federally  supported  R&D  could  help  to  protect 
regional  and  local  environments,  reduce  demands  on 
energy  and  various  natural  resources,  and  improve  the 
delivery  of  state  and  local  services. 

In  recent  years,  federal  funding  of  R&D  for  the  civilian 
sector  has  been  growing  rapidly.  It  is  now  in  excess 
of  $7  billion  annually.  But  its  impact  on  meeting 
public  expectat ions--on  filling  the  everyday  needs 
of  the  people--often  seems  disappointing. 

A feeling  is  now  developing  along  the  lines  that  inter- 
governmental action  in  science  and  technology  must 
become  more  of  a two-way  flow.  More  initial  state  and 
local  involvement  in  setting  federal  R&D  agendas  appears 
to  be  one  way  of  generating  this.  Governors,  mayors, 
state  legislators,  and  county  and  local  officials  have 
far  better  ideas  of  the  problems  and  the  needs  of  their 
communities  than  do  Washington  officials.  They  should 
have  more  of  an  input  into  the  decision-making  that 
results  in  federal  R&D  budgets  in  the  civilian  sector. 

A related  problem  is  that  much  federally  generated 
R&D  that  might  be  applicable  to  public  use  on  a state 
or  local  level  is  not  adequately  assessed  or  demon- 
strated. As  a result,  its  usefulness  cannot  be 
properly  evaluated.  In  many  cases,  research  that  might 
ultimately  serve  a public  purpose  is  not  carried  far 
enough  into  application,  implementation  or  the  federal 
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commercialization  stage.  This  situation  could  be  im- 
proved by  more  attention  and  closer  cooperation  between 
federal  and  state  and  local  officials  concerning  research 
ut i 1 i zat ion . ^ 

It  is  from  this  perspective  that  the  technology  transfer  process 
emanates.  A clearly  existing  need  and  a potential  solution  source. 

The  speakers  today  represent  the  various  facets  of  the  current  tech- 
nology transfer  process  and  will  be  addressing  their  respective 
portions  to  provide  you  with  a cohesive  overall  picture  complete 
with  problems,  needs,  and  accomplishments. 

The  scenario  of  the  session  has  been  so  constructed  as  to  provide 
you  with  exposure  to  different  levels  involved.  To  begin  with,  the 
first  speaker  will  set  the  stage  by  describing  the  objectives  and 
functioning  of  the  Federal  Laboratory  Consortium,  one  of  the  most 
important  national  technical  networks.  We  will  then  proceed  to  the 
heart  of  the  technology  transfer  process  by  hearing  from  a local 
Science  Advisor.  As  a further  amplification  of  depicting  "how" 
technology  transfer  works,  we  will  then  hear  described  the  appli- 
cation techniques  of  one  Federal  agency.  Representing  the  require- 
ments side  will  be  a speaker  from  one  of  the  oldest  and  best  regional 
innovation  groups  in  the  country--the  California  Innovation  Group 
(ClG)--and  a Deputy  City  Manager  from  San  Diego  addressing  the 
Urban  Consortium.  Finally,  for  an  overall  perspective  and  a vision 
of  the  future,  we  will  close  with  the  Director  of  Intergovernmental 
Programs,  National  Science  Foundation. 

As  one  added  stimulant,  I have  asked  Dr.  Kay  Inaba,  President  of 
XYZYX  Information  Corporation  of  Los  Angeles,  to  provide  us  with  his 
reactions  in  a summary  of  the  session. 

Our  speakers  are  ready,  you  have  had  an  opportunity  to  become 
familiar  with  their  backgrounds  from  the  handouts  you  received  and 
my  introductions  will  therefore  be  as  brief  as  possible.  A limited 
number  of  questions  or  comments  will  be  entertained  at  the  conclusion 
of  each  with  a portion  at  the  end  of  the  session  reserved  for  general 
d i scuss i on . 

It  remains  also  to  be  said  that  this  entire  process  as  you  will  hear 
is  really  rather  new  and  its  future  is  before  it.  It  is  our  hope 
that  by  discussing  issues  and  developing  recommendations  at  such 
major  conferences  as  this  one  today,  we  can  together,  contribute  to 
its  ultimate  future  and  direction  for  our  common  benefit. 
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SECONDARY  UTILIZATION  OF  RESEARCH,  OR  TECHNOLOGY  TRANSFER  AND  THE  R&D 
LABORATORIES  ~~ 


Mi‘.  Geovga  Linsteadt 
Chairman 

Federal  Laboratory  Consortium  for  Technology  Transfer 


INTRODUCTION 

What  are  the  barriers  to  research  utilization?  How  can  we  better 
utilize  our  research  products?  What  are  the  reasons  for  less  than 
optimal  utilization  of  research  efforts?  These  questions  are  asked 
not  only  by  DoD  but  also  by  other  Federal  government  agencies.  DoD 
developed  research  products  are,  in  one  way  or  another,  intended 
for  use  in  the  nation's  defense.  Therefore,  it  seems  only  natural 
that  the  principal  concern  of  this  symposium  should  be  improvement 
of  research  utilization  within  the  military  services. 

However,  others,  outside  of  DoD,  could  also  benefit  from  existing 
military  research  efforts.  Many  military  R&D  "spin-offs"  have  found 
their  way  into  the  private  sector  and  become  household  items.  Often 
these  spin-off  products  just  happened;  they  were  not  the  primary 
product  of  a research  effort.  Perhaps  other  existing  research 
products  developed  by  and  for  DoD  could  be  systematically  shared 
with  other  Federal  agencies  or  state  and  local  government  agencies. 

Such  a transfer  is  a form  of  research  ut i 1 i zat i on--perhaps  the 
ultimate  in  DoD  research  utilization. 

If  the  results  of  DoD  research  can  be  utilized  by  other  government 
agencies,  then  the  original  DoD  research  investment  can  be  seen  as 
having  contributed  over  and  above  its  primary  task  of  satisfying  a 
military  need.  Secondary  research  utilization,  or  technology  transfer, 
has  attracted  the  attention  of  many  individuals  and  organizations  as  a 
logical  method  of  enhancing  the  productive  output  of  research  efforts. 

The  transfer  of  DoD  research  to  other  Federal,  state  and  local  agencies 
can  be  provided  through  the  department's  many  laboratories.  These 
laboratories,  such  as  the  Navy  Personnel  RSD  Center,  contain  a vast 
reservoir  of  technology,  plus  facilities,  equipment  and  capable 
professional  people. 

Two  facts  justify  making  the  resources  of  these  laboratories  avail- 
able to  other  Federal,  state  and  local  government  agencies: 

1.  The  resources  of  these  laboratories  could  provide  solutions 
to  many  of  the  problems  faced  by  these  other  agencies. 

2.  The  secondary  utilization  of  research  results  presents  the 
taxpayer  with  a greater  return  on  his  investment  in  science 
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and  technology.  Enornious  sums  of  taxpayers'  monies  go  to 
research  and  development,  and  every  effort  must  be  made  to 
ensure  effective  primary  and,  where  possible,  multiple  use 
of  that  investment.  In  today's  inflationary  environment,  this 
is  an  absolute  necessity. 


THE  FEDERAL  LABORATORY  CONSORTIUM  FOR  TECHNOLOGY  TRANSFER 

Of  course,  secondary  research  utilization  or  technology  transfer  is 
more  easily  said  than  done.  The  Federal  laboratories  are  presently 
accountable  to  many  Federal  government  agencies  and  no  integrating 
management  system  exists  over  these  laboratories  to  make  certain  that 
the  secondary  technology  transfer  and  research  utilization  process 
is  fully  productive.  A voluntary  organization,  the  Federal  Laboratory 
Consortium  for  Technology  Transfer,  has  been  established  to  coordinate 
the  technology  transfer  efforts  of  its  members.  To  date,  70  of  the 
largest  Federal  government  laboratories  and  centers  from  a number  of 
high  technology  agencies  are  participating. 

The  Federal  Laboratory  Consortium  emerged  from  and  is  patterned 
after  the  Department  of  Defense  Technology  Transfer  Consortium 
which  was  established  in  July  1971.  At  that  time,  representatives 
from  eleven  DoD  laboratories  met  at  the  Naval  Weapons  Center  to 
discuss  the  potential  ability  of  DoD  laboratories  to  transfer  military 
oriented  technology  to  other  government  agencies  for  nonmilitary 
applications.  There  are  now  kl  DoD  facilities  involved  in  Consortium 
activities.  The  previous  DoD  Consortium  and  the  newly  emerged  Federal 
Technology  Transfer  Consortium  have  received  administrative  support 
from  the  Naval  Material  Command  and  the  National  Science  Foundation. 
The  purpose  of  the  Consortium,  as  stated  in  its  operating  policy, 
i s as  fol 1 ows : 

The  Department  of  Defense  laboratories  are  a source  of  tech- 
nology for  the  solution  of  those  civil  sector  problems  which 
are  amenable  to  technological  solutions.  The  primary  role 
of  the  in-house  laboratories  is  to  provide  a research  and 
development  base  for  the  development  ot  systems  required 
to  fulfill  the  national  security  mission  of  the  Department 
of  Defense.  However,  these  laboratories  can  serve  a 
vital  secondary  role  in  the  adaptation  of  technology  to 
other  fields  and  areas  of  need  to  the  extent  that  it  does 
not  adversely  impact  on  the  primary  Department  of  Defense 
mission.  A consortium  of  Department  of  Defense  labora- 
tories is  formed  for  the  purpose  of  coordinating  inter- 
actions with  other  Federal  agencies  and  technology  users 
at  the  Federal,  state,  and  local  level,  and  of  coordinating 
the  efforts  in  this  endeavor.  The  Technology  Transfer 
Consortium  is  an  association  of  Department  of  Defense 
laboratories  working  together  through  an  informal  affilia- 
tion. The  main  thrust  of  the  consortium  activity  is 
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through  the  individual  and  cooperative  efforts  of  the 
laboratories  involved,  with  an  emphasis  on  the  transfer 
and  adaptation  of  technology  through  person-to-person 
mechan i sms . 

THE  EVOLUTION  OF  POLICY 

Many  recent  policy  and  analytical  milestones  have  paved  the  way 
for  the  current  expansion  of  the  Federal  technology  transfer  acti- 
vities. The  National  Science  and  Technology  Policy,  Organization 
and  Priorities  Act  of  1976  states,  "...The  Federal  Government  should 
support  and  utilize  engineering  disciplines  and  make  maximum  use  of 
the  engineering  communi ty. . .advise  and  assist  the  Director  in 
identifying  and  fostering  policies  to  facilitate  the  transfer  and 
utilization  of  research  and  development  results  so  as  to  maximize 
their  application  to  civilian  needs...." 

Likewise,  President  Ford,  in  a report  to  Congress  in  February  1975, 
declared:  "One  of  the  clear  lessons  of  the  past  few  years  is  that 
our  society,  as  well  as  that  of  the  rest  of  the  world,  is  intimately 
tied  to  technology  and  the  science  that  produces  it.  Certainly  our 
own  approaches  to  problems  in  energy,  environment,  food  production, 
and  the  well-being  of  the  national  economy  will  include  substantial 
contributions  from  science  and  technology.  As  a nation  we  are 
fortunate  to  have  an  extraordinary,  strong  science  and  technology 
base  to  draw  on  in  dealing  with  these  and  other  important  problems. "5 

The  Committee  on  Federal  Laboratories  of  the  Federal  Council  for 
Science  and  Technology,  in  a report  to  the  Office  of  Management  and 
Budget  in  March  197^,  recommended  tnat  "...consistent  with  existing 
laws  and  with  established  relationships  with  private  industry, 
universities  and  nonprofit  institutions,  existing  research  and 
development  capabilities  in  Federal  establishments  be  utilized 
effectively  to  define  and  solve  technological  problems  and  guide 
the  technical  content  of  policy  decisions  relating  to  such  urgent 
national  needs  as  the  environment,  transportation,  and  health. 

Studies  by  the  Federal  Council  on  Science  and  Technology  in  197^^ 
and  by  the  Council  of  State  Governments  in  1973^  found  a high  potential 
for  bringing  federally  developed  science  and  technology  to  bear  on 
the  operations  and  performance  of  state  and  local  governments.  These 
same  studies  noted  serious  barriers  to  the  effective  application  of 
such  technology. 

A December  1972  report  from  the  General  Accounting  Office®  recommended: 

1.  A government-wide  policy  for  technology  transfer  with  guide- 
lines issued  to  Federal  agencies  to  implement  a formal, 
active  technology  transfer  process; 
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2.  That  the  Secretary  of  Defense  establish  policy  and  procedures 
to  encourage  more  extensive  application  of  existing  defense 
technology  to  civilian  problems;  and 

3.  The  establishment  of  a technology  transfer  consulting  team 

as  a central  focus  to  help  Federal  agencies  match  technological 
resources  with  pressing  needs. 

The  Action  Conference  on  Intergovernmental  Science  and  Technology 
Policy,  held  in  June  1972  at  Harrisburg,  Pennsylvania,  and  attended 
by  over  two  hundred  leaders  from  both  the  public  and  private  sectors, 
made  a number  of  resolutions.  They  called  for  (1)  strengthening 
of  state  and  local  government  through  applied  science  and  engineering, 
(2)  Federal -state-local  partnerships  in  applied  research  and  develop- 
ment, and  (3)  using  Federal  laboratories  in  domestic  programs. 5 

The  President's  March  1972  message  to  Congress  on  science  and  tech- 
nology also  noted  that  "Federal  research  and  development  activities 
generate  a great  deal  of  new  technology  which  could  be  applied  in 
ways  which  go  well  beyond  the  immediate  mission  of  the  supporting 
agency.  I believe  the  government  has  a responsibility  to  transfer 
the  results  of  its  research  and  development  activities  to  wider  use 
in  the  private  sector. 


PROBLEMS  IN  TRANSFERRING  FEDERAL  LABORATORY  TECHNOLOGY 

Laboratories  within  the  various  Federal  government  agencies  have 
diverse  missions  which  reflect  the  missions  of  the  parent  agencies. 

It  is,  therefore,  not  surprising  that  those  agencies  established 
for  the  specific  purpose  of  serving  the  people  in  rather  obvious 
ways  should  have  technology  transfer  effort  written  into  their 
missions.  However,  the  Congress,  viewing  DoD  primarily  as  a defense 
establishment,  is  inclined  to  limit  activities  of  the  Department  to 
those  related  to  defense.  During  times  of  relative  peace,  the 
natural  viewpoint  of  the  people  and  the  need  for  tight  financial 
constraints  tend  to  limit  DoD  activities.  Unfortunately,  this 
happens  at  a time  when  the  Department  is  in  the  best  position  to 
provide  a useful  service  to  the  people  by  making  defense-generated 
technologies  available  for  use  by  the  general  public.  This  has  been 
the  aim  of  DoD's  technology  transfer  efforts  and  many  successes  have 
been  achieved  both  by  the  Department  of  Defense  and  the  Consortium. 
But,  there  is  still  much  to  accomplish. 

Before  addressing  the  various  constraints  on  the  intergovernmental 
use  of  DoD  laboratories,  it  should  be  mentioned  that  legal  support 
for  such  use  already  exists  in  the  form  of  the  Economy  Act  of  1932 
and  the  more  recent  Intergovernmental  Cooperation  Act  of  1968. 

These  legal  precedents  exist  to  allow  DoD  facilities  to  assist  state 
and  local  governments.  However,  focus  of  mission  agencies  and 
appropriations  provided  them  do  not  permit  large  formal  undertakings. 


A report  by  the  Council  of  State  governments  ^ points  out  some  of  the 
severe  constraints  that  hamper  the  technology  transfer  efforts  of 
the  Federal  laboratories: 

1.  Budget  Limitations.  No  funds  are  budgeted  for  technical 
assistance  to  state  and  local  governments.  This  lack  of 
funds  means  that,  for  all  practical  purposes,  the  policy 

of  intergovernmental  technology  transfer  can  be  implemented 
only  if  (a)  the  state  or  local  government  can  reimburse  or 
(b)  the  laboratory  is  persuaded  to  sacrifice  some  part  of 
its  core  activity. 

2.  Statutory  Restrictions.  In  the  case  of  DoD  laboratories,  the 
so-called  Mansfield  Amendment  prohibits  the  use  of  R6D 

funds  for  any  research  project  or  study  unless  there  is  a 
potential  for  relationship  to  a military  function  or  need. 

This  permits  cooperative  R6D  projects  having  civilian  benefits 
only  if  they  can  also  be  shown  to  be  relevant  to  the  national 
security  mission.  While  the  Mansfield  Amendment  is  considered 
by  some  as  a potential  barrier  to  a more  extensive  and  pro- 
ductive utilization  of  the  spin-off  potential  of  defense 
laboratories,  it  is  in  fact  less  of  a problem  in  the  near-term 
than  employment  and  money  limitations. 

3.  Policy  Inadequacies.  Few  Federal  agencies  have  an  affirmative 
action  policy  to  encourage  their  R&D  centers  to  diversify 
their  roles  and  services  on  behalf  of  state  and  local  govern- 
ments. Still  fewer  have  requested  budgetary  resources  or 
manpower.  Nor  have  they,  for  the  most  part,  sent  out  a 
signal  to  state  and  local  governments  that  the  principal 
laboratories  are  being  designated  as  technology  support 
centers . 

k.  Information  Gaps.  Despite  the  genuine  desire  of  many  Federal 
R6D  centers  to  apply  their  resources  to  state  and  local 
governmental  problems,  there  is  a general  lack  of  information 
and  data  on  available  laboratory  skills.  The  Council  of  State 
Governments  recommends  that,  as  a first  step  in  bringing 
the  laboratories  together,  a clear  definition  be  compiled 
on  the  capabilities  of  the  respective  laboratories.  The 
laboratories  can  thereby  define  what  they  have  to  offer  in 
response  to  user  needs  and  what  they  believe  they  can  do 
in  addressing  such  specific  problems  as  air  quality  standards 
or  instrumentation  for  monitoring  changes  in  water  quality. 

5.  Commun i cat i ons . The  problem  of  communication  between  local 
government  officials  and  laboratory  researchers  has  also 
been  a block  in  attempts  to  involve  laboratories  in  state 
act i vi t i es . 

6.  Technology  Adaptation  Capabilities.  Frequently,  simply 
providing  a given  technological  response  to  a state  or 
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local  problem  is  not  sufficient.  More  than  likely,  the  user 


needs  technical  help  to  adapt  or  modify  the  laboratory's 
knowledge.  Few  Federal  laboratories  are  in  any  position 
to  provide  this  service  to  state  and  local  governments. 


7.  Competition  with  Business.  This  issue  could  become  important 
should  the  Federal  R&D  centers  move  significantly  toward 
providing  contract  services  to  state  and  local  governments 
or  Federal  civilian  agencies  for  research  and  problem-solving. 
The  Council  of  State  Governments  recommends  that  a policy 
position  be  implemented  which  would  enable  the  laboratories 
to  provide  at  least  first-stage  services  to  state  and  local 
governments,  and  also  set  boundary  conditions  effectively 
limiting  the  laboratories  from  extending  themselves  into 
full-service  providers  of  public  technology. ^ 

SUMMARY 

In  conclusion,  many  statements  have  been  made  during  the  past  few 
years  indicating  that  obtaining  the  maximum  return  from  our  research 
investment  is  a worthwhile  goal.  However,  to  derive  this  maximum 
benefit  from  the  nation's  investment  in  research,  primary  technology 
must  also  be  applied,  if  possible,  toward  a secondary  application. 

The  Federal  government  invests  large  sums  of  money  in  research 
annually,  and  means  must  be  found  to  better  utilize  this  technology 
through  improved  secondary  technology  transfer. 

The  Federal  RED  laboratories  represent  a large  technology  resource 
which  may  be  applied  to  civilian  needs.  These  laboratories,  located 
throughout  the  nation  and  operating  on  annual  budgets  totalling 
in  the  billions  of  dollars,  can  supply  technical  assistance  and 
management  support  to  those  state  and  local  governments,  and  in 
some  cases,  private  industries  attempting  to  solve  the  nation's 
problems.  Many  of  these  laboratories  are  actively  seeking  to  share 
their  facilities  and  expertise  with  public  and  private  agencies 
having  compatible  requirements.  The  Department  of  Defense  Tech- 
nology Transfer  Consortium,  as  a subset  of  the  Federal  Laboratory 
Consortium  for  Technology  Transfer,  is  actively  participating  in 
this  endeavor. 

As  far  as  policy  is  concerned,  there  do  not  appear  to  be  any  barriers 
to  DoD  laboratories  providing  assistance  to  state  and  local  govern- 
ments. The  President's  message  in  March  1972  provided  good  reason 
for  intergovernmental  cooperation  in  the  use  of  science  and  tech- 
nology, and  the  General  Accounting  Office  and  the  Office  of 
Management  and  Budget  have  supported  this  principle. 
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TECHNOLOGY  TRANSFER  IN  ACTION 


Mr.  Gerald  E.  Milter 
Technology  Transfer  Coordinator 
State  of  Oregon 


In  reading  the  announcement  of  this  conference,  I noticed  that  the 
intent  is  to  increase  the  utilization  of  research  data  produced  by 
R&D  Laboratories  of  the  military  services.  Certainly  the  title 
and  topic  is  appropriate  for  I would  suggest  that  at  no  time  in  the 
history  of  the  Federal  research  community  has  there  been  greater 
pressure  by  Congress  and  by  the  public  for  maximizing  utilization  of 
our  research  efforts.  Judging  from  the  quality  of  previous  speakers 
and  from  the  attendance,  it  appears  that  those  of  you  here  feel  the 
same  way.  From  the  program  outline  it  is  obvious,  as  it  should  be, 
that  the  emphasis  of  this  conference  is  on  the  greater  utilization 
of  the  research  data  produced  and  used  by  the  Department  of  Defense. 

But  those  of  us  on  this  panel  are  here  today  to  tell  you  of  potential 

new  users  which  probably  most  of  you  have  considered  before. 

As  you  know,  I am  also  a Federal  scientist,  employed  by  the  United 
States  Navy  since  1969.  But  I am  somewhat  unique  in  that  I have, 
for  the  last  six  years,  devoted  full  time  to  finding  civilian  uses 
of  the  technology  produced  by  the  Federal  laboratories,  both  military 
and  nonmilitary.  The  ultimate  goal  of  my  job  is  to  use  this  in- 
formation toward  solving  problems  faced  by  the  cities,  counties  and 
states  in  the  United  States.  Thus,  I have  been  born  of  the  innocence 
of  the  lamb  at  the  Federal  level  and  subsequently  fed  to  the  lions 
at  the  local  level.  I do  not  use  the  term  to  describe  the  political 

and  economic  environment  that  is  faced  by  the  local  officials  in 

running,  on  a day-to-day  basis,  the  cities,  counties  and  states  in 
which  you  and  I live.  And  believe  me,  it  is  an  entirely  different 
ball  game  in  that  area  than  you  and  I,  as  Federal  scientists  in 
Federal  laboratories,  are  exposed  to. 

As  an  example,  let  me  cite  several  situations  which  you  may  not  be 
aware  of.  I was  recently  told  that  in  the  Boston  Fire  Department, 
the  average  retirement  age  is  slightly  over  AO  years  and  three- 
fourths  of  all  firemen  that  retire  in  Boston  do  so  under  disability 
retirement,  not  service  longevity  retirement.  This  means  that  the 
city  of  Boston  has  to  budget  literally  millions  of  dollars  to  pay 
for  individuals  who  are  on  disability  retirement  and  yet  who  have 
years  of  potentially  productive  life  ahead  of  them.  In  Oklahoma, 
eight  cents  out  of  every  dollar  spent  on  the  collection  of  urban 
refuse  pays  for  the  liability  insurance  each  city  must  have  since 
collecting  garbage  and  trash  is  the  most  hazardous  duty  any  city 
employee  can  perform.  In  fact,  in  some  of  the  smaller  communities 
in  Oklahoma  the  insurance  premiums  cost  more  than  the  cost  of 
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actually  collecting  the  refuse.  In  Poway,  California,  a small 
community  northeast  of  San  Diego,  topographical  conditions  prevented 
the  fire  department  from  communicating  with  their  fire  trucks  in 
certain  parts  of  the  city.  In  Klamath  Falls,  Oregon,  five  firemen 
have  suffered  heart  attacks  during  the  last  two  years.  All  have 
survived;  all  are  drawing  disability  pay.  In  Nashville,  Tennessee, 
two  police  officers  are  killed  within  a six-week  span.  One  stabbed 
to  death;  one  shot  by  a 17  year-old  youth  with  a .22  caliber 
pistol.  Neither  officer  was  wearing  body  armor  which  is  highly 
available  and  relatively  inexpensive.  Aside  from  the  tragic  human 
loss,  the  City  of  Nashville  is  out  over  $100,000  for  widow  payments 
and  medical  costs.  In  Multnomah  County,  Oregon,  the  most  populated 
county  in  Oregon,  the  nearest  good  aggregate  for  road  construction  is 
now  over  35  miles  from  the  asphalt  plant.  The  added  haulage,  plus 
increased  cost  of  oil  and  labor  has  significantly  increased  the  cost 
of  repairing  city  streets.  And  finally,  approximately  six  weeks 
ago,  a small  state-owned  bridge  along  the  southern  coast  of  Oregon 
collapsed.  Fortunately,  no  one  was  injured,  but  the  bridge  collapsed 
because  the  wood  pilings  that  supported  it  had  been  eaten  through 
by  marine  borers  and  this  damage  was  not  visible  to  the  casual 
observer  or  the  naked  eye.  But  this  collapse  now  forces  the  State 
Highway  Department  to  seek  a method  of  determining  the  structural 
soundness  of  all  other  wood  pilings  of  state-owned  bridges  in  Oregon. 
If  done  manually,  this  could  cost  the  State  several  million  dollars 
a year  alone.  I could  continue  to  cite  literally  hundreds  of 
examples  of  problems  which  are  daily  confronting  city,  county, 
and  state  governments  in  the  United  States.  This  is  why  I'm 
here  today  to  talk  to  you  about  what  you  people  can  do  to  help 
local  government  reduce,  or  at  least  hold  in  line,  these  costs. 

Now,  let's  go  back  and  take  some  of  these  examples  to  show  you 
how  research,  knowledge  and  information  generated  by  Federal 
agencies  has,  in  fact,  assisted  local  governments  already.  In  the 
case  of  the  Klamath  Falls,  Oregon  Fire  Department,  information 
gathered  on  physical  fitness  training  programs  in  other  fire 
agencies,  and  brought  together  by  the  National  Bureau  of  Standards, 
was  directly  applicable  to  the  potential  establishment  of  a physical 
fitness  training  program  for  the  Klamath  Falls  Fire  Department. 
Further,  information  generated  by  the  National  Fire  Research 
Administration  was  used  to  verify  that  such  physical  fitness  programs 
could  reduce  heart  attacks  in  firemen.  This  makes  the  program  much 
more  acceptable  to  the  community,  to  the  firemen  themselves  and  to 
the  City  Council.  In  the  case  of  the  police  officers  in  Nashville, 
Tennessee,  the  second  death  resulted  in  an  immediate  demand  by 
the  mayor  that  body  armor  be  provided  to  all  law  enforcement 
personnel  in  the  city.  The  technology  advisor  in  that  city  put  in 
an  urgent  call  to  the  Department  of  Defense,  via  a technology 
transfer  contact,  and  considerable  research  data  generated  by  the 
United  States  Army  was  provided  to  the  city  of  Nashville  at  no 
cost.  In  addition,  the  information  used  by  the  San  Diego  Police 
Department,  which  had  been  previously  generated  with  the  assistance 
of  the  United  States  Army,  was  also  forwarded  to  Nashville  and  a 
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complete  body  armor  specification  was  prepared  in  three  days  and 
delivered  to  the  mayor.  I am  told  that  today,  all  750  sworn  officers 
in  the  Nashville,  Tennessee,  Police  Department  daily  wear  body  armor. 
Closer  to  home,  the  fire  chief  of  Poway,  California,  after  hearing 
of  the  interest  of  the  Federal  labs  in  helping  solve  local  problems, 
contacted  the  Naval  Ocean  Systems  Center  (NOSC)  here  in  San  Diego  for 
assistance.  Their  problem  was  basically  one  of  radio  propagation 
and  certainly  if  the  Navy  or  the  Department  of  Defense  had  any 
technology  appropriate  to  local  government,  the  study  of  radio  wave 
propagation  problems  was  one  clearly  adaptable  to  local  government. 

A quick  check  with  technology  experts  here  at  this  laboratory 
indicated  that  they  could,  in  fact,  offer  technical  advice  to  the 
Fire  Department.  Topographic  maps,  tower  heights,  power  output  of 
the  transmitter  and  other  technical  data  were  requested  and  furnished 
by  the  Poway  Fire  Department  in  a matter  of  24  hours.  Four  days 
later,  NOSC  provided  a written  report  to  the  Poway  Fire  Department 
describing,  in  complete  detail,  how  to  solve  their  problem  by  the  use 
of  repeater  sites  on  appropriate  hills.  1 might  mention  that  this 
was  done  by  a single  scientist  from  NOSC  at  home  on  his  own  time 
and  he  was  absolutely  ecstatic  to  be  able  to  offer  his  assistance 
to  the  community  in  which  he  lived.  Subsequently,  the  Commanding 
Officer  received  a letter  of  appreciation  from  the  Fire  Chief  and 
a note  that  funds  had  been  requested  from  the  City  Council  to 
implement  the  recommendations  provided  by  NOSC. 

In  the  case  of  the  Oregon  bridge  collapse,  I can't  offer  such  an 
outstanding  success  story  because  we  have  found  that  there  is  little 
existing  information  that  would  be  directly  appropriate  to  that 
problem.  But  what  we  have  found,  and  I think  it's  extremely 
appropriate  for  this  conference,  is  that  the  National  Bureau  of 
Standards  which  houses  a large  nondestructive  testing  research 
group,  has  said  that  they  believe  this  program  could  be  of  signi- 
ficant national  interest  and  that  they  might  be  able  to  adapt  their 
present  research  program  to  include  nondestructive  testing  of 
bridge  pilings.  You  probably  have  noted  that  the  examples  given 
are  more  hard-technology  oriented,  and  as  I understand,  you  are 
involved  in  peop 1 e- re  1 ated  research.  But  local  government  has  a 
lot  of  those  problems  also.  Alcohol  counseling  programs,  produc- 
tivity improvement,  employee  training,  job  stress  for  police  and 
firemen,  personnel  performance  measures,  and  selection  criteria 
for  policemen  are  all  problems  I have  been  given  through  the  years. 

I am  well  aware  that  most  of  us  in  the  room  today  do  not  have  the 
kind  of  flexibility  potentially  exhibited  by  NBS  with  the  bridge 
piling  problem.  Frankly,  I am  not  asking  that  you  change  your 
mission  or  change  your  R&D  projects.  What  I am  asking  is  that  you 
make  an  effort  to  let  local  government  know  what  data  you  already 
have  existing.  I am  further  suggesting  that,  unless  you  are  working 
in  an  exotic  field  such  as  lasers,  destruction  of  orbiting  satellites, 
or  some  other  type  of  highly  speculative  technology,  the  chances 
that  your  research  results  may  have  civilian  application  are  good. 
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What  I would  like  to  see  is  you  people  making  an  effort  to  see  that 
it  is  utilized  at  the  local  level.  Assuming  that  you  have  that 
interest,  allow  me  to  give  you  some  thoughts  about  assisting  local 
government . 

Perhaps  the  most  significant  advice  that  I can  offer  is  that  the  local 
government  officials  near  your  base  or  your  federal  agency  will  never 
come  to  you  for  help.  I think  this  is  for  two  principal  reasons: 

First,  local  officials  never  anticipate,  particularly  if  you  come 
from  a DoD  laboratory,  that  you  are  interested  in  working  with  local 
government.  You  must  let  them  know  that.  Second,  the  majority  of 
local  officials  which  I have  met  find  it  hard  to  believe  that  a 
Federal  agency  has  anything  to  offer  to  the  local  government. 

Most  of  the  local  officials  with  which  I have  dealt  conceive  of  a 
Federal  scientific  laboratory  as  a place  where  all  those  exotic, 
neat  things  are  done,  but  which  they  would  never  understand  in  a 
million  years.  Again,  it  is  up  to  you  people  to  dispel  that  myth. 

So,  what  can  you  do  specifically  to  enhance  the  utilization  of  your 
research  data  by  local  government?  First,  you  have,  or  will  shortly 
hear,  the  discussion  by  Mr.  Linsteadt  of  the  Federal  Laboratory 
Consortium.  Certainly,  your  participation  and  cooperation  with  that 
consortium  gives  you  a step-up  in  the  potential  utilization  of 
your  research  data  from  your  particular  laboratory  and  I would 
heartily  encourage  your  joining  or  at  least  participating  with  the 
Federal  Laboratory  Consortium.  But  whether  you  do  or  not  should  not 
limit  you  in  taking  the  second  step  and  that  is  making  yourself 
known  to  the  local  governments  in  which  your  laboratory  resides.  I'm 
referring  specifically  to  the  city  manager,  the  mayor,  or  any  other 
principal  city,  county  or  state  administrator.  Go  to  them,  explain 
who  you  are  and  why  you  are  there.  Simply  tell  them  you  would  like 
to  share  your  information  and  help  them  solve  problems  which  are 
facing  their  communities.  Now,  1 must  warn  you  that  you  will  be 
received  initially,  I suspect,  with  some  skepticism.  Local 
officials  are  inundated  with  consultants  and  others  who  offer  cure- 
alls  for  local  problems.  Further,  most  local  officials  feel  their 
problems  are  not  of  a scientific  nature  and  thus  will  find  it  difficult 
to  initially  understand  what  you  have  to  offer.  Finally,  city 
officials  are  confronted  on  a daily  basis  with  labor  crises,  citizens' 
complaints,  financial  problems,  etc.,  so  that  it  is  very  likely  that 
they  will  forget  your  initial  offer.  This  means  you  must  be  pre- 
pared to  go  back  several  times.  But  please,  don't  give  up. 

When  you  do  talk  with  the  local  official,  ask  him  what  kinds  of 
problems  are  facing  his  community.  See  if  you  have  any  projects  on- 
going at  your  laboratory  which  might  be  appropriate.  Be  willing  to 
provide  reports,  phone  consultations,  personal  on-site  visits,  if 
appropriate  and  possible,  because  nothing  works  as  well  in  this 
business  as  the  one-on-one  discussion.  And  once  the  local  official 
finds  out  that  you  are  sincere  and  can,  in  fact,  contribute,  the 
chances  that  your  research  data  will  continue  to  be  utilized  are 
greately  enhanced. 
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There  is  one  thing,  however,  which  you  should  not  do  and  that  is 
approach  the  local  community  officials  emphas i z i ng  that  you  are  a 
scientist.  Instead,  approach  them  as  a member  of  the  community  with 
possibly  some  unique  contribution.  That  is  why  it  is  extremely 
important  that  you  take  your  research  data  and  adapt  it  to  the 
city  problems.  In  other  words,  find  out  what  the  problems  are  and 
offer  solutions.  Do  not  go  in  with  the  solutions  and  look  for  the 
problems.  Now,  it  is  likely  that  your  technology  will  not  be 
directly  appropriate  and  adaptable  to  the  city  problem.  It  might 
take  some  modification,  or  interpretation,  and  that  is  where  you, 
as  an  intermediary  between  the  Federal  laboratory  and  the  local 
government,  can  serve  a very  useful  function. 

Up  to  this  point  you  may  be  saying,  "Gee,  that  sounds  okay  to  me, 
but  it  also  sounds  like  he's  talking  about  hours  of  my  time,  I 
just  don't  have  it."  Frankly,  that  is  not  true.  It  is  absolutely 
amazing  to  me  how  much  help  can  be  provided  in  a relatively  few 
minutes  of  your  time  each  week  and  I think  you  will  find  that  once 
you  make  those  initial  contacts,  you  will  realize  the  potential 
that  can  be  gathered  from  your  laboratory. 

Finally,  one  always  asks,  "What's  in  it  for  me?"  Well,  I guess  I 
could  ask  the  same  thing  of  the  70  laboratories  that  presently 
comprise  the  all  volunteer  Federal  consortium,  particularly  those 
that  are  in  the  Department  of  Defense  because  as  you  all  know,  there 
is  no  charter  in  DOD  requiring  us  to  support  local  government.  I 
guess  I could  ask  that  same  thing  of  the  electronic  engineer  who 
so  graciously  helped  the  City  of  Poway.  I will  be  honest  to  say 
that  there  will  be  no  kudos  from  the  Department  of  Defense  and  there 
will  be  probably  very  little  from  your  laboratory  management. 

However,  I think  I can  offer  two  rewards.  First  of  all  my  experience 
has  been,  while  serving  as  a technology  transfer  coordinator  for 
NOSC  that  the  working  level  individual  in  the  laboratory  is  very 
eager  to  finally  find  somebody  who  really  wants  to  use  the  results 
of  his  research. 

Furthermore,  he  is  able  to  see  a direct  result  of  his  efforts. 
Secondly,  there  is  in  fact  an  intrinsic  satisfaction  that  can  be 
realized  from  knowing  that  you're  solving  real  world  problems, 
particularly  in  the  community  in  which  you  live  and  pay  taxes.  It 
is  my  personal  opinion  that  the  concept  of  technology  transfer  will 
someday  be  accepted  at  the  Federal  level,  and  that  in  fact,  there 
will  be  a Federal  technology  transfer  program.  Until  that  day 
arrives,  however,  I personally  believe  in  the  concept,  and  feel 
that  those  of  us  who  are  scientists  and  produce  scientific  research 
data  can  realize  a personal  satisfaction  in  finding  greater  utili- 
zation of  that  data  by  applying  those  results  to  problems  in  our 
own  community.  I have  worked  with  approximately  150  cities  in  38 
different  states  since  I got  into  the  technology  transfer  business 
back  in  1972.  As  a result,  I believe  I can  honestly  say,  as  one  of 
the  comedians  used  to  say  on  television,  that  if  you  make  the  effort 
to  find  greater  utilization  of  your  research  data  at  the  local  level. 
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"your  mother  will  thank  you,  your  father  will  thank  you,  I will  thank 
you,  and  I know  damn  well  that  the  local  government  will  thank  you." 


SCME  APPROACHES  TO  TRANSFERRING  FEDERAL  TECHNOLOGY  TO  STATE  AND  LOCAL 
GOVERNMENT:  THE  LAWRENCE  LIVERMORE  LABORATORY  EXPERIENCE 


I 
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Mv.  Charles  F.  Miller 
Program  Manager,  Technology  Apglioations 
Lawrence  Livermore  Laboratory 
University  of  California 


INTRODUCTION 


Since  its  inception,  the  Lawrence  Livermore  Laboratory,  operated  by 
the  University  of  California  for  the  Energy  Research  and  Development 
Administration,  has  supported  technology  transfer  efforts.  Initially, 
these  efforts  were  under  the  auspices  of  the  Atomic  Energy  Commission. 
The  AEC's  programs  relating  to  technology  transfer  grew  from  an 
early  narrow  policy  of  permitting  only  nuclear-related  interactions 
to  one  encompassing  energy  research  of  all  kinds.  The  scope  of 
these  transfer  activities  reached  full  flower  with  the  formation  of 
the  Energy  Research  and  Development  Administration  (ERDA)  in  January 
1975. 


ERDA's  legislative  authority  states  "...the  Administration  shall 
disseminate  scientific,  technical  and  practical  information  acquired 
through  information  programs  and  other  appropriate  means,  and  shall 
encourage  the  dissemination  of  scientific,  technical  and  practical 
information  relating  to  energy  so  as  to  enlarge  the  fund  of  such 
information  and  to  provide  that  free  interchange  of  ideas  and 
criticism  which  is  essential  to  scientific  and  industrial  progress 
and  public  understanding."'' 

Additionally,  ERDA  has  been  mandated  to  reach  national  energy  goals 
as  soon  as  is  feasible  through  research,  development,  test,  and 
demonstrations  involving  cooperative  efforts  with  industry  and  with 
other  public  bodies.  To  successfully  develop  new  sources  of 
energy  or  to  develop  energy  conserving  technologies  requires  that 
technology  transfer  be  an  integral  part  of  ERDA's  program  develop- 
men  t . ' ^ 

Beyond  those  programs  with  inherent  technology  transfer  elements, 
ERDA  is  committed  to  achieving  maximum  utility  of  all  technologies 
arising  from  its  research  activities.  Each  ERDA  Laboratory  is 
encouraged  to  support  efforts  to  spinoff  specific  and  useful  ERDA 
technologies  from  the  Laboratory  to  industry,  and  to  state  and  local 
governments.  The  technologies  are  diverse  in  that  they  consist  of 
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ideas,  hardware,  processes,  special  facilities,  technical  projects, 
developments,  and  individual  expertise. 


At  the  Lawrence  Livermore  Laboratory  (LLL)  , the  Technology  Appli- 
cations Group  (tag)  is  responsible  for  these  activities.  Over 
the  past  few  years  a variety  of  approaches  have  been  used  to  help 
meet  the  challenge  of  delivering  a very  broad  "product  line"  to 
an  extremely  varied  set  of  "customers."  The  purpose  of  this  paper 
is  to  describe  briefly  some  of  these  activities  directed  toward 
state  and  local  governments  and  to  present  some  conclusions  based 
on  the  lessons  we  learned. 


SELECTED  STATE-WIDE  PROJECTS 

In  most  areas  of  the  United  States,  various  public  programs  support 
activities  designed  to  provide  technical  assistance  or  advice  to 
members  of  state  and  local  governments 3, 1 ^ In  August  1975,  a 
one-year  experiment  was  begun  to  test  methodologies  of  transfer 
to  state  and  local  governments  with  the  State  of  Hawaii  as  the  test 
location.  Before  this  experiment  had  concluded,  ERDA's  San  Francisco 
Operations  Office  requested  that  we  perform  a similar  task  in  the 
State  of  Arizona.  The  Hawaii  experiment  has  concluded  '5  and  the 
Arizona  effort  continues. 


THE  HAWAII  EXPERIMENT 

Six  visits,  each  for  about  one  week,  were  made  to  the  State  of  Hawaii 
with  the  objective  of  developing  and  testing  a methodology  of  tying 
Federal  sources  of  technology  to  civilian  receptors,  with  LLL  serving 
as  a broker  or  "linker."'^  The  State  of  Hawaii  was  selected  as  the 
location  because  it  represents  a separate,  self-contained,  socio- 
economic system  for  which  data,  technological  needs,  and  results  of 
efforts  could  be  identified  in  a short  period  of  time.  Additionally, 
the  time  and  cost  constraints  associated  with  communicating  with  the 
mainland  tend  to  discourage  the  use  of  technical  resources  outside 
of  the  State. 

The  initial  visits  were  devoted  to  meeting  the  appropriate  people, 
to  gaining  an  understanding  of  the  structure,  needs,  history, 
institutions,  and  strengths  of  the  state,  and  to  establishing 
working  relationships  with  key  people  at  the  State  and  city/county 
level.  Probably  as  much  as  one-third  of  our  time  was  spent  on 
these  efforts  before  sufficient  rapport  and  confidence  were 
established  to  effect  technology  transfers. 

The  strategy  was,  first,  to  establish  credibility  by  providing 
useful  information  to  requestors,  then  to  effect  transfers  of  ERDA 
energy  technologies,  and,  finally,  to  encourage  and  abet  institu- 
tional relationships  between  the  State  and  mainland  resources. 


I 

I 
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In  addition  to  visits  by  LLL  personnel,  a number  of  other  activities 
were  conducted.  The  Department  of  the  Navy,  with  National  Science 
Foundation  assistance,  presented  a two-day  Public  Works  Management 
and  Maintenance  Seminar  in  Honolulu  in  response  to  expressed  needs. 

The  seminar  was  effective  and  well  received.  Later,  three  Technology 
Transfer  Workshops  were  held,  one  each  in  Hilo,  Maui,  and  Honolulu. 

The  workshops,  directed  toward  representatives  of  city/county 
governments  and  local  industry,  were  conducted  by  technology  transfer 
representatives  of  three  Navy  Laboratories  in  California,  the 
California  Innovation  Group,  and  LLL.  We  also  assisted  the 
University  of  Hawaii  in  planning  and  presenting  a one-day  Solar 
Energy  Workshop. 

These  face-to-face  meetings  and  workshops  led  to  a large  volume 
of  correspondence  between  the  Hawaii  participants  and  the  mainland 
representatives.  Significantly,  since  the  end  of  the  experiment 
(and  the  end  of  face-to-face  contacts)  the  frequency  of  correspondence 
has  dwindled  to  an  occasional  letter  among  a few  of  the  key  people 
i nvol ved . 

Our  overall  goals  seem  to  have  been  met.  The  awareness  level  of  key 
Hawaii  people  of  mainland  resources  of  technical  information  has 
been  increased;  "useful"  transfers  have  taken  place;  and,  with  the 
assignment  of  a full-time  employee  to  a new  ERDA  Honolulu  office 
and  a strong  tie  to  a state-wide  office,  an  important  step  has 
been  taken  toward  an  institutional  tie  to  mainland.  Federal  technology. 


THE  ARIZONA  PROJECT 

To  help  carry  out  his  responsibilities  within  Federal  Region  IX,  the 
manager  of  ERDA's  San  Francisco  Operations  Office  appointed  as  his 
representative  to  the  State  of  Arizona,  Mr.  Don  Pearman,  Deputy 
Director,  ERDA  Los  Angeles  Office.  Mr.  Pearman,  following  several 
meetings  with  officials  of  state  and  local  governments,  requested 
that  we  assist  him  in  his  Arizona  responsibilities.  The  request 
came  from  his  desire  to  offer  a broader  base  of  technical  assistance 
to  the  various  institutions  in  the  State  and  his  feeling  that  our 
activities  in  technology  transfer  could  bear  fruit  in  Arizona. 

Again,  the  initial  marketing  efforts  began:  meet  the  key  people; 
gain  an  understanding  of  their  backgrounds,  needs,  constraints;  learn 
and  understand  the  local  structures,  hierarchies;  identify  and  become 
acquainted  with  ready  receptors.  These  activities  brought  us  into 
contact  with  the  State  capitol,  county  governments,  city  governments, 
and  officials  of  the  Navajo  and  Hopi  Indian  Nations.  As  in  Hawaii, 
the  plan  was  to  establish  credibility  by  attempting  to  address  the 
broad  spectrum  of  technical  needs,  to  establish  an  awareness  of 
other  Federal  sources  of  available  technology,  to  effect  significant 
transfers  of  ERDA  technology,  and  to  help  create  self-sustaining 
linkages  between  state  and  local  governments  and  federal  technologies. 
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To  date,  the  project  appears  to  be  progressing  satisfactorily. 

Solid  connections  have  been  made  with  active  receptors  and  dialogues 
have  been  initiated  between  Arizona  personnel  and  other  Federal 
laboratories.  The  Navy's  Public  Works  Management  and  Maintenance 
Seminar  will  be  presented  in  Flagstaff  this  summer  for  Public  Works 
personnel  in  Northern  Arizona.  The  Naval  Weapons  Center,  China  Lake, 
has  arranged  for  the  loan  of  instruments  to  help  acquire  data  on 
several  solar  projects  within  the  state.  Our  Laboratory  is  planning 
several  solar  energy  workshops  in  the  state  and  is  developing  energy 
projects  with  the  Hopi  nation.  Within  the  year,  the  need  for  frequent 
face-to-face  meetings  will  diminish  as  the  technical  relationships 
solidify. 


TECHNOLOGY  TRAINING  PROGRAM 


In  another  approach  to  Technology  Transfer,  LLL's  Mechanical  and 
Electronic  Engineering  Departments,  with  initial  funding  from  the 
National  Science  Foundation,  established  TTP  — the  Technology 
Training  Program.  Starting  in  September  1975,  198  students  repre- 
senting 91  different  organizations  have  been  trained  at  LLL  in 
three  areas  of  technology.'^  Three  courses  were  developed  (micro- 
computers, digital  logic,  and  welding  and  bonding  technology)  and  two 
of  these  have  each  been  offered  three  times. 


The  courses  are  offered  at  no  charge  to  the  participants,  but  each 
participant's  institution  is  responsible  for  personal  expenses 
such  as  travel  costs,  meals,  and  lodging.  The  lecture  portions  of 
the  courses  are  presented  in  a specially  designed  classroom  contained 
in  a double-wide  trailer  module.  The  trailer  forms  a 24  x 60  foot 
room  with  space  for  32  students.  Television  cameras  and  two  monitors 
enhance  students'  visibility  of  the  instructor  and  visual  aids  and 
cameras  record  the  entire  lecture  on  video  tape.  The  courses  also 
feature  hands-on  training  in  laboratories,  custom  fitted  for  each 
course,  in  a building  adjacent  to  the  classroom.  Carefully  pre- 
pared workbooks  are  also  supplied  to  the  students. 

To  measure  the  effectiveness  of  these  courses,  several  feedback 
pathways  are  used.  Each  student  participates  in  two  surveys. 

During  the  first  class  day,  a questionnaire  is  completed  which 
evaluates  students'  expectations.  At  the  end  of  the  course,  another 
questionnaire  allows  the  students  to  evaluate  the  course  in  terms 
of  their  pre-course  expectations.  Results  of  pre-  and  post-course 
surveys  have  resulted  in  appropriate  modifications  to  course 
material,  working  areas,  or  presentations.'® 

Another  Important  measurement  of  the  effectiveness  of  these  courses 
is  a series  of  follow-up  studies  of  selected  organizations  whose 
employees  have  attended  the  course.  The  studies  include  visits  to 
the  organization  and  interviews  with  former  students.  Some  benefits 
to  participating  organizations  have  been  reported. '9 
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Some  identified  results  are  listed  below; 


1.  To  improve  efficiencies,  the  California  Department  of  Water 
Resources  is  using  a microcomputer  data  collection  system  at 
the  Oroville  Dam  to  relay  information  directly  from  the  dam 
to  the  control  center  in  Sacramento,  California. 

2.  As  a result  of  the  Welding  and  Bonding  course.  Bay  Area  Rapid 
Transit  District  (BART)  maintenance  personnel  have  adopted 
techniques  estimated  to  save  thousands  of  dollars. 

3.  A physician  with  the  Kaiser  Hospital,  Oakland,  California,  has 
developed  a prototype  interface  between  a Coulter-S  Blood 
Counter  and  an  LLL-designed  microcomputer  which  creates  a 
system  to  automatically  flag  abnormally  high  or  low  blood  values. 

4.  Engineers  of  the  San  Francisco  Bay  Area's  East  Bay  Municipal 
Utility  District  are  applying  advanced  weld  evaluation  techniques 
to  determine  if  segments  of  a 50-year  old  pipeline  can  be 
repaired  or  must  be  replaced. 

5.  California  Department  of  Water  Resources  personnel  have  applied 
TTP-taught  welding  techniques  to  repair  eroded  water-pump 
impellers  at  a savings  in  time  and  money. 

6.  The  City  of  San  Jose,  California,  has  applied  microcomputer 
technology  to  the  design  and  specification  of  traffic  con- 
trollers with  estimated  annual  savings  of  $75,000. 

Potentially  more  important  than  results  reported  by  former  students 
may  be  the  multiplier  effect  of  these  courses.  Video  tapes  and  course 
materials  are  available  for  loan  so  that  former  students  may  present 
the  courses  to  others  in  their  own  establishment.  For  example, 

Mr.  Monroe  Postman,  the  Public  Technology,  Inc.,  technology  agent  for 
the  City  of  San  Jose,  borrowed  video  tapes,  workbooks,  and  other 
materials  from  LLL.  He  then  trained  17  San  Jose  employees  in 
microcomputer  technology  and  applications.  In  addition  to  the 
traffic  controller  application  earlier,  the  city  employees  will  apply 
this  newly  gained  knowledge  to  minicomputers  to  be  installed  in 
the  city's  library  system.  Presently,  over  35  organizations  have 
borrowed  video  tapes  and  course  material  and  an  additional  20 
organizations  are  on  a wait  list.  Ultimately,  literally  thousands 
of  technical  employees  will  benefit  from  this  training. 


CONCLUSIONS 

Experiences  with  an  active  (as  opposed  to  reactive)  technology 
transfer  program  at  LLL  have  led  to  a number  of  conclusions  about 
the  necessary  ingredients  and  steps  to  ensure  successful  transfer  of 
Federal  technology  to  state  and  local  governments.  These  conclusions 


are  reinforced  by  our  experiences  in  transferring  technology  to 
industry  and  by  reports  from  other  Federal  laboratories. 

The  most  important  conclusions  are  the  following: 

1.  The  technology  transfer  activity  must  be  a full-time,  fully 
funded  and  directed  effort  on  the  part  of  the  technology 
source. 

2.  Without  active,  informed  and  enthusiastic  technology  receptors 
in  the  state  and  local  government,  transfer  efforts  will 
fail. 

3.  Technology  transfer  agents,  in  the  field  or  in  the  office, 
must  have  access  to  a broadly  based  body  of  technical  in- 
formation and  experience,  such  as  The  Federal  Laboratory 
Consortium  for  Technology  Transfer. 

4.  These  agents  must  have  the  freedom  and  the  motivation  to 
aggressively  seek  opportunities  and  to  respond  satisfactorily 
and  in  a timely  manner  to  all  requests  for  assistance. 

5.  Person-to-person  contacts,  over  a long  period  of  time,  between 
sources  and  receptors  in  the  field  are  essential. 

6.  Merely  providing  information  in  the  form  of  reports  is  usually 
not  sufficient  to  effect  transfers.  Often,  additional  develop- 
ment work  (tailoring  a solution  to  a problem)  and/or  training 
the  receptor  in  the  use  of  a technical  fix  is  required. 

7.  The  transfer  of  a technology  will  be  completed  when  the 
technology  becomes  generally  accepted  practice,  or  when  the 
chief  officer  of  a governmental  unit  routinely  assesses 
available  technology  when  presented  with  a problem,  or  when 
the  technology  is  readily  available  in  the  marketplace. 

8.  The  transfer  of  Federal  technologies  to  state  and  local 
governments  is  an  integrating  process,  involving  considerable 
effort  on  the  part  of  the  receptor  as  well  as  the  source  and 
sometimes  involving  assistance  from  other  sources,  receptors, 
or  technology  "brokers." 

Recently,  we  read  the  written  testimony  of  Mr.  Joseph  W.  Smollen,  III, 
presented  to  the  U.  S.  Senate's  Subcommittee  on  Aerospace  Technology 
and  National  Needs,  Committee  on  Aeronautical  and  Space  Science  at 
hearings  held  September  22,  23,  and  24,  1975.^  Mr.  Smollen  was 
at  that  time  a NASA  employee  serving  as  an  advisor  to  the  Regional 
Planning  Commission  for  Jefferson,  Orleans,  St.  Bernard,  and  St. 
Tammany  Parishes,  Louisiana,  under  Title  IV  of  the  Intergovernmental 
Personnel  Act.  In  his  testimony,  Mr.  Smollen  offered  "A  Few  Comments 
on  Technology  Transfer"  which  so  closely  followed  our  experiences 
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that  we  thought  it  appropriate  to  close  with  these  thoughts  (for 
"NASA,"  of  course,  one  can  read  "ERDA,"  "DOT,"  "DoD,"  "ERA,"  etc.): 

1.  It's  much  more  difficult  in  actual  practice  than  in  the  fine 
brochures  and  polished  presentations  on  the  subject. 

2.  There  are  no  experts  in  technology  transfer--only  varying 
degrees  of  ignorance. 

3-  Full-time  men-on-the-scene  are  necessary. 

A.  Fast  response  and  technical  back-up  are  required.  Local  and 
state  agencies  don't  care  about  the  internal  Federal  budget 
and  justification  cycles--they  just  want  results  or  signs  of 
progress  now.  In-depth  technical  experts  need  to  be  ready 
to  catch  the  next  plane  to  talk  with  the  local  people.  The 
local  officials  also  don't  care  about  Federal  travel  budgets 
and  justifications. 

5.  Knowledge  of  the  local  state  people,  government,  culture, 
history,  and  trends  is  imperative.  People  still  distrust 
"outsiders."  You  can  be  the  best  technologist  in  the  world-- 
and  if  you  don't  understand  what  is  important  to  the  local 
populat ion--you  will  do  more  harm  than  good. 

6.  The  local  elected  and  appointed  leaders  must  be  progressive 
and  want  innovation  in  government.  And  the  local  leaders 
(not  the  NASA  man)  should  conduct  the  press  conference  when 
you  have  results  to  show. 

7.  Progress  and  results  will  be  painfully  slow  under  the  best 
of  conditions. 

8.  Data,  information,  and  information  systems  may  be  the  most 
important  "product"  transferred. 

9.  Not  all  technology  transferred  is  NASA  technology.  What  is 
important  is  the  solution  to  the  problem  at  hand  whether  it 
involves  technology  or  not. 

10.  A lot  of  the  technology  is  not  carried  far  enough  by  the  federal 
laboratories  to  be  cheap  enough  for  city  and  state  governments 
to  afford  i t . 

11.  Much  of  the  Federal  technology  is  not  put  in  a form  or  format 
that  is  recognized  as  useful  to  the  local  government. 

12.  There  are  few  financial  incentives  for  cities  to  experiment 
with  new  technology. 
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Industry  has  no  great  incentives  to  pursue  new  technology 
to  the  point  of  commercial  feasibility  or  market  aggregat ion-- 
especially  with  the  cities. 

Once  you  are  successful  with  a few  things  in  a location,  you 
begin  to  be  inundated  with  phone  calls,  visiting  firemen,  and 
stacks  of  mail--to  the  point  that  it  almost  interferes  with 
your  basic  mission  and  overwhelms  your  mind  with  obscure 
facts.  But  buried  in  the  mail  and  visitors  is  the  occasional 
"pearl"--that  will  help  solve  one  of  your  local  problems-- 
or  where  you  can  help  someone  else.  The  problems  of  success 
are  more  pleasant  than  those  of  failure--but  they  are  no  less 
difficult  to  deal  with. 

There  will  be  more  failures  than  successes  at  first.  The 
most  tragic  failure  is  to  quit  trying. 

The  best  technology  transfers  are  done  quietly  with  little 
fanfare . 

A lot  of  good  tries  will  fall  through  the  crack. 

You  must  have  "godfathers"  in  local,  state  and  NASA  organiza- 
tions or  you  can't  survive  to  do  the  job. 

Talk  is  cheap  in  technology  transfer. 

The  customer  (local  and  state  government)  is  (almost)  always 
right. 

If  you  can't  put  an  existing  working  device  or  product  in  the 
local  man's  hands,  then  don't  talk  about  it  until  you  can. 


TECHNOLOGY  TRANSFER,  A CASE  STUDY  - LOCAL  GOVERNMENT 


Mr.  D.  M.  Delaharre 

President,  California  Innovation  Group,  Inc. 


On  behalf  of  the  California  Innovation  Group,  I would  like  to  express 
our  appreciation  for  the  opportunity  to  come  together  with  all  of 
you  to  exchange  ideas  on  how  we  may  improve  our  individual  technology 
utilization  programs.  We  have  many  issues  which  need  to  be  addressed 
in  the  area  of  technology  transfer  and  utilization.  I will,  however, 
be  speaking  primarily  of  the  technology  transfer  and  utilization 
process  and  how  we  may  elevate  our  current  "affair"  with  the  Federal 


-137- 


laboratories  to  a legal  marriage.  I feel  it  would  be  useful  to 
spend  a few  minutes  summarizing  the  background  and  evolution  of  the 
California  Innovation  Group.  Additionally,  I would  like  to  give  you 
a few  of  my  personal  views  concerning  science  and  technology  transfer 
and  why  I feel  the  CIG's  and  related  projects  are  so  essential  to  the 
future  health  of  our  cities. 

First,  let  me  say  that  for  our  purposes,  we  view  the  term  "technology 
transfer"  to  mean  "Learning  to  use  available  materials,  techniques 
and  processes  to  improve  our  effectiveness  and  save  money."  Most  of 
what  I have  to  say  today  is  aimed  at  trying  to  make  that  happen 
for  us  in  local  government. 

It  is  our  opinion  that  the  majority  of  the  problems  we  run  up 
against  are  not  really  the  technical  problems  associated  with  transfer 
or  the  application  of  high  technology,  but  the  political  and  administra- 
tive aspects  of  government.  Some  dramatic  failures  have  been  noted 
in  efforts  to  implement  a technology  transfer  activity  and  most  of 
these  can  be  related  directly  to  personality  problems  or  unwillingness 
on  one  side  or  the  other  to  make  any  adaptations  in  communicating  with 
each  other.  Commitment  by  people  such  as  yourself  and  those  in  local 
government  who  believe  in  what  we  are  all  about  here  today,  can 
overcome  those  problems  and  allow  the  technical  people  to  operate 
in  a much  more  efficient  manner.  Commitment  and  desire,  however,  will 
not  do  it  without  some  mechanism  or  structure  for  us  to  work  within. 

We  feel  that  explaining  the  history  and  evolution  of  CIG  may  help 
in  sorting  out  a more  comprehensive  plan  of  attack. 

THE  CALIFORNIA  FOUR  CITIES  PROGRAM 

The  California  Innovation  Group  is  actually  the  result  of  an  experiment 
which  was  started  in  September  of  1971,  known  as  the  California  Four 
Cities  Program.  The  original  participating  cities  in  California  were 
San  Jose,  Fresno,  Pasadena  and  Anaheim. 

The  original  concept  consisted  of  putting  together  a team  of  partici- 
pants that  were  considered  essential  to  providing  technology  transfer 
for  the  cities.  The  primary  sponsor  for  that  initial  program  was  the 
National  Science  Foundation,  with  the  National  Aeronautics  and  Space 
Administration  an  associate  sponsor,  providing  program  administration 
and  support  through  the  Jet  Propulsion  Laboratories  in  Pasadena.  The 
two  remaining  elements  of  the  team  were  the  four  cities  and  the 
sponsoring  aerospace  firms  (Northrop,  S.A.I.,  Aero  Jet  and  Lockheed). 

Program  organization  during  the  first  three  years  of  our  program  was 
quite  informal.  A contract  agreement  was  made  between  the  National 
Science  Foundation  and  JPL  with  subsequent  contracts  directly  to  the 
Industrial  firms  providing  the  Science  Advisors.  Typically,  the 
Science  Advisors  reported  directly  to  the  City  Manager,  and  his 
industrial  back-up  site  provided  support  to  his  activities.  The 
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' Science  Advisors  came  from  aerospace  firms  which  were  to  act  as 

primary  resources  in  the  transfer  of  technology  to  cities. 

Although  the  Four  Cities  Program  established  quite  a lengthy  list 
of  objectives  to  be  achieved,  I think  it  is  fair  to  say  that  the 
primary  objective  was  to  provide  a feasibility  test  to  determine  the 
[ acceptability  and  usability  of  a high  technology  transfer  agent 

housed  within  local  government  management.  Generally  speaking, 
the  results  obtained  during  those  first  three  years  of  our  existence 
have  been  well  accepted  by  both  the  sponsoring  agencies  and  the  City 
personnel . 

At  the  conclusion  of  that  initial  experiment,  the  United  States 
General  Accounting  office  was  asked  to  evaluate  the  program,  and  in 
a report  to  Congress,  concluded,  and  I quote:  "The  program  generally 
achieved  the  specific  program  objectives  and  showed  that  such 
partnership  arrangements  can  bring  about  improvements  in  operating 
local  governments  through  technological  innovations."  We  should  also 
mention  the  fact  that  the  cost  of  providing  these  Science  Advisors 
has  continually  increased  while  the  per  city-dollar  financing  from 
the  National  Science  Foundation  has  decreased. 

In  summary,  the  principal  objective  of  the  program,  that  of  proving 
the  viability  and  usefulness  of  a Science  Advisor,  was  clearly  shown, 
and  we  still  believe  that  the  Science  Advisor  is  the  most  important 
element  of  both  programs. 


THE  CALIFORNIA  INNOVATION  GROUP 

Building  upon  the  success  of  the  initial  Four  Cities  experiment,  the 
California  Innovation  Group  was  established.  The  initial  expansion 
of  the  Four  Cities  Program  resulted  in  the  Cities  of  Santa  Clara, 
Burbank,  Santa  Ana,  Garden  Grove,  Huntington  Beach,  and  San  Diego 
joining  the  original  four  cities.  Since  that  time,  the  City  of  Brea 
has  also  been  added.  In  our  expansion  efforts,  it  became  obvious 
that  if  we  were  to  truly  represent  a statewide  activity,  the  League 
of  California  Cities  would  be  an  absolute  necessity  to  have  as  a 
partner  in  this  technology  transfer  effort. 

The  primary  objective  of  this  expansion  effort  was  to  develop  a more 
"program  oriented"  activity,  and  rather  than  operating  as  individual 
projects  in  each  city,  we  would  utilize  an  intergovernmental  approach 
The  specific  objectives  developed  for  the  California  Innovation  Group 
are  quite  lengthy  and  I will  not  go  through  them  in  detail.  However, 
they  can  be  categorized  into  several  general  areas.  They  include  the 
development  of  the  team  approach  and  the  intergovernmental  aspects  of 
such  an  approach;  the  establishment  of  a statewide  technology 
communication  network  through  the  California  League  of  Cities;  the 
identification  and  implementation  of  the  necessary  steps  to  in- 
stitutionalize the  activity,  which  I will  go  into  very  shortly;  the 
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expansion  of  our  involvement  and  linkage  with  programs  such  as  the 
Urban  Consortium,  the  Urban  Technology  System,  and  Public  Technology 
Inc.,  in  general,  and  other  networks  interested  in  providing  technology 
transfer  assistance  to  cities;  and  finally,  it  was  our  objective  to 
develop  a more  diversified  resource  base  with  less  dependence  on  the 
National  Science  Foundation. 

Our  basic  approach  to  achieving  these  objectives  I will  go  into  in  some 
detail  later.  However,  we  continued  the  successful  aspects  of  the 
Four  Cities  Program  and  have  established  our  statewide  capability 
through  the  League.  From  the  point  of  view  of  the  participating 
city  managers,  we  can  safely  say  that  the  results  obtained  during  the 
past  two  years  of  the  California  Innovation  Group's  existence  have 
clearly  demonstrated  the  value  of  such  transfer  capabilities,  and 
we  have  established  the  necessary  mechanisms  to  take  on  various  types 
of  programs  ranging  from  test  and  evaluation  to  proof  of  concept  for 
new  systems  or  hardware.  Simply  stated,  we  have  gone  from  an  experi- 
ment to  an  operating  support  function  for  the  participating  cities. 

For  the  purposes  of  this  meeting,  1 believe  a brief  explanation  of 
the  evolution  of  the  CIG  organization  may  be  worthwhile.  One  of  the 
problems  identified  with  the  original  Four  Cities  experiment  was  the 
lack  of  active  program  management.  The  very  nature  of  the  experiment 
resulted  in  a very  passive  role  being  taken  by  NASA/JPL  to  determine 
the  worth  of  the  individual  Science  Advisors.  This  approach,  although 
acceptable  for  the  initial  phase  of  the  program,  did  not  seem  to 
provide  the  type  of  local  government  control  that  would  bring  the 
program  into  clear  focus  with  our  overall  needs.  For  this  reason, 
the  participating  city  managers  elected  to  select  an  individual  to 
act  as  Principal  Investigator/Program  Manager,  and  more  or  less  work 
directly  for  the  cities. 

Originally  established  as  an  informal  consortium,  the  California 
Innovation  Group  was  not  a legal  entity.  As  a result,  it  was 
necessary  to  establish  a pass-through  agency  to  monitor  and  disburse 
NSF  grant  funds.  This  was  accomplished  through  the  Institute  for 
Local  Self  Government,  a non-profit  corporation  associated  with  the 
League  of  California  Cities. 

We  tried  to  establish  more  effective  lines  of  communication  with 
Public  Technology,  Inc.  (PTI)  and  the  Federal  Laboratory  Consortium 
(FLC) . The  FLC  support  has  been  very  effective  and,  in  fact,  they 
have  established  a California  Federal  Consortium,  made  up  of  lA 
laboratories  located  in  the  state,  working  directly  with  our  program 
office  and  the  individual  Science  Advisors.  George  Linsteadt 
will  be  speaking  on  the  FLC  later;  however,  I would  like  to  say  that 
through  the  efforts  of  these  Federal  Laboratory  Technology  Utilization 
personnel,  the  FLC  has  emerged  as  a primary  resource  to  CIG.  It  is 
our  desire  to  expand  our  partnership  and  meetings  such  as  this  one 
today  will  surely  help.  The  League  of  California  Cities  provides 
a full-time  field  representative  devoting  her  time  to  science  and 
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technology  dissemination  activities.  I might  note  at  this  point  that 
the  dissemination  coordinator  for  the  League  i's  available  to  all  other 
programs  including  Federal  Laboratory  Technology  Utilization  activities 
and  cities  in  the  state,  not  just  the  CIG  participants. 

The  next  evolution  the  CIG  went  through  was  that  of  becoming  a 
California  non-profit  corporation.  This  action  has  provided  us  with 
the  necessary  status  as  a legal  entity  and  organizational  structure 
of  sufficient  strength  to  function  in  a statewide  or  regional 
capacity.  A Board  of  Trustees  comprised  of  private  sector  people 
was  formed  and  the  city  managers,  previously  serving  on  the  Policy 
Board,  became  a local  government  Advisory  Board. 

We  have  learned  from  the  California  Innovation  Group  that  individual 
cities  require  a slightly  different  or  iiKidified  organizational 
approaches  within  their  individual  political  structures.  Generally, 
the  Science  Advisors  report  directly  to  the  City  Manager.  San  Diego 
has  established  a separate  office  called  San  Diego  Technology  Action 
Center  (SANDTAC) . Allan  SJoholm,  the  Science  Advisor  there,  is  on  an 
IPA  assignment  from  the  Navy  Personnel  R&D  Center,  San  Diego. 

I hope  that  this  explanation  of  our  organizational  evolution  has 
given  you  some  insight  into  the  various  problems,  and  hopefully, 
some  potential  solutions  to  those  problems  that  arise  in  the  develop- 
ment of  such  an  organization. 

I thought  it  might  be  worthwhile  for  purposes  of  this  meeting  to  very 
briefly  note  the  primary  elements  of  our  work  breakdown  structure. 

They  are  the  ongoing  functions  of:  (1)  Technology  Analysis  (which 
includes  (a)  problem/needs  update,  (b)  technology  resources  analysis, 
and  (c)  task  force  support);  (2)  Technology  Utilization  (which 
includes  (a)  problem  solution  analysis,  (b)  alternatives  selection 
and  implementation,  (c)  technology  brokerage,  and  (d)  utilization 
documentation);  and  (3)  Information  Network  (which  includes  (a) 
technology  dissemination,  (b)  CIG  Handbook,  and  (c)  information 
exchange);  and  (4)  Special  Projects  (which  includes  (a)  product 
development,  and  (b)  evaluation  and  test  of  existing  products). 

We  feel  that  these  on-going  functions  really  must  be  supported  and 
nurtured  by  the  program  office,  the  individual  Science  Advisors  and 
outside  resources  such  as  the  Federal  Laboratory  Consortium.  In 
addition,  we  have  special  projects  that  basically  fall  into  two 
categories:  (!)  Product  Development  or  Proof  of  Concept,  and 

(2)  Evaluation  and  Testing  of  Existing  Products. 

Very  briefly,  I would  like  to  explain  the  program  support  that  we've 
been  able  to  generate  for  the  California  Innovation  Group.  The 
prime  sponsor  is  the  National  Science  Foundation,  with  contributing 
sponsors  in  the  financial  area  such  as  FEA,  HUD.  ERDA,  and  local 
government  cash.  Also  very  essential  to  the  program  are  the  non- 
financial  contributors  (including  Aerospace  Industries,  DOT,  Federal 
Laboratory  Consortium,  University  of  California,  League  of  California 
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Cities,  NLC/USCM,  FBI)  in  the  form  of  staff  time,  equipment  on  loan, 
brokerage  service,  and  all  different  types  of  activities.  We've 
had  varying  degrees  of  success  with  these  support  agencies.  We've 
run  into  certain  problems  with  the  University  system  in  that, 
generally  speaking,  it  needs  to  be  a well-defined,  specific  project 
before  we  have  had  any  degree  of  success.  One  obvious  reason  for 
this  is  that  our  work  is  generally  not  very  sexy  from  an  academic 
point  of  view.  One  resource  that  we  have  been  able  to  tap  and 
very  effectively  use  is  that  of  the  Federal  laboratories,  as  I 
mentioned  before. 

In  summary,  I'd  like  to  ramble  just  a little  bit  and  give  you  a few 
of  our  thoughts  about  what  this  all  means  to  local  governments  in- 
volved in  the  California  Innovation  Group.  The  mechanism  of  CIG 
has  been  able  to  develop  an  effective  team  of  participants  from 
both  the  public  and  private  sectors  to  provide  technology  transfer 
assistance  to  local  governments.  This  team  consists  of  local 
government  people  themselves,  high  technology  firms,  support  from 
the  League  of  California  Cities  and,  perhaps  most  important,  the 
individual  Science  Advisor  who  resides  in  each  one  of  our  partici- 
pating jurisdictions.  I have  mentioned  that,  coupled  with  this 
team,  is  the  technical  resource  of  the  Federal  Laboratory  Consortium. 
We  feel  that  the  efforts  of  CIG  have  had  positive  results  in  bringing 
this  national  technical  resource  into  a posture  of  accessibility  to 
local  government.  I believe  we  have  learned  that  institutionalization 
of  an  activity  such  as  CIG  requires  considerable  exterior  stimuli 
and  resources.  The  basic  problem  here,  as  we  all  know,  stems  from 
the  difficulty  in  effecting  change  in  government.  That  change  has 
been  taking  place  within  the  CIG  cities,  and  we  feel  maybe  to  a 
lesser  degree  in  cities  which  come  in  contact  with  the  program 
through  the  League  of  California  Cities.  We  must  also  effect  change 
within  the  Federal  Government  including  the  vast  resources  of  the 
Fp  '.°tal  Government  including  the  vast  resources  of  the  Federal 
l.aDoratory  System.  The  results  of  the  California  Innovation  Group 
to  date  clearly  indicate  that  large  cost  savings  or  cost  avoidance 
can  be  achieved  through  the  applications  of  programs  like  ours. 

One  of  the  serious  problems  associated  with  expounding  on  the  success 
of  that  program  is  that  it  becomes  more  and  more  difficult  to  explain 
why  we  feel  that  continued  federal  assistance  is  necessary.  That 
is  why  we  must  also  consider  our  failures.  The  activities  associated 
with  technology  transfer  inherently  carry  some  degree  of  risk.  As 
a result,  CIG  and  other  transfer  programs  need  the  luxury  of  failure. 
This  is  not  possible  without  outside  support.  You  all  know  that 
cities  are  in  the  forefront  of  visibility  and  pressure  from  the  in- 
dividual taxpayers.  Risk  money  of  any  kind  is  difficult  to  come  by 
at  the  local  level.  A good  example  of  this  problem  was  a recent 
demonstration  that  CIG  cosponsored  with  the  U.  S.  Conference  of 
Mayors  and  NASA  to  demonstrate  a satellite-based  teleconferencing 
system.  As  it  turned  out,  the  demonstration  was  exceptionally 
successful;  however,  if  it  had  gone  the  other  way,  if  it  had 
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gone  badly,  it  would  have  been  a National  Science  Foundat ion/C I G 
\ program.  Both  of  these  organizations  could  survive  that  type  of 

failure  since  it  represents  the  very  nature  of  the  enterprise.  It 
might  not  have  been  that  easy  to  explain  if  local  government  tax  money 
was  used  to  finance  the  demonstration  and  it  had  gone  badly.  We 
need  that  once-removed  identity  that  CIG,  PT I , and  other  innovation 
groups  provide  to  protect  the  cities  and  people  who  enter  into  these 
^ innovation  experiments. 

• We  feel  that  the  efforts  to  establish  national  networks  are  heading 

in  the  right  direction.  We  want  to  be  part  of  that.  In  fact,  we 
hopefully  will  provide  a leadership  role  on  the  west  coast  to 
establish  these  national  networks.  Additionally,  we  feel  that  some 
type  of  federal  assistance  on  an  institutionalized  basis  has  got  to 
be  developed.  Currently,  the  National  Science  Foundation  cannot 
provide  that  kind  of  assistance,  since  they  are  restricted  to  experi- 
mental projects  of  a limited  duration.  We  are  not  sure  if  an  inter- 
agency approach  should  be  pushed  or  some  other  arrangement.  It  is 
possible  that  the  Federal  Regional  Councils  could  take  on  the 
responsibility  of  sponsoring  regional  technology  transfer  activities; 
however,  the  interest  has  not  been  expressed  to  date. 

In  closing,  I would  extend  the  offer  that  CIG  is  willing  to  work 
with  all  of  you  in  developing  a national  network.  We  are  convinced 
that  if  we  are  to  achieve  the  type  of  return  on  investment  we  talk 
about  wanting,  it's  going  to  take  a lot  of  working  together.  Every 
new  technology  transfer  activity  we  become  involved  with  has  expanded 
our  own  capabilities. 

THE  NATIONAL  CONSORTIUM  FOR  TECHNOLOGY  INITIATIVES 


Mr.  John  Loakwood 
Deputy  City  Manager 
City  of  San  Diego /National  Consortium 
for  Technology  Initiatives 


I'm  really  wearing  two  hats  today.  As  an  employee  of  the  City  of 
San  Diego,  I'd  like  to  welcome  you  to  San  Diego,  and  while  you're  here 
if  there  are  any  city  facilities  you'd  like  to  visit  or  anything  else 
that  our  office  can  do  to  assist  you,  we'd  certainly  like  to  try. 

So,  please  call  us.  Secondly,  I'd  like  to  take  just  a minute  or  two 
to  discuss  with  you  the  National  Consortium  for  Technology  Initiatives. 
What  is  it?  Four  years  ago  approximately,  a group  of  local  government 
officials  got  together  to  discuss  the  problem  of  transferring 
technology  between  the  cities  and  counties  of  the  United  States.  The 
problem  is  obvious;  cities  were  working  on  problems,  they  all  had 
problems,  but  the  technology  transfer  vehicle  just  wasn't  there. 


-1A3- 


Out  of  those  discussions  and  with  some  financial  assistance  from 
Bob  Crawford  and  our  friends  at  the  NSF , also  DOT  and  HUD,  the 
National  Consortium  for  Technology  Initiatives  was  formed.  That's 
a different  consortium  from  the  one  you  were  just  hearing  about.  It 
consists  of  the  28  cities  in  the  United  States  who  have  populations 
in  excess  of  500,000  persons  and  six  large  urban  counties.  Collec- 
tively, it's  big  business.  They  spend  over  $20  billion  a year;  they 
have  over  one  million  employees.  The  Secretariat  for  the  National 
Consortium  is  PTI,  which  is  a non-profit  corporation  based  in 
Washington,  D.  C.,  sponsored  by  the  U.  S.  Conference  of  Mayors- 
National  League  of  Cities  governmental  jurisdictions. 

The  Consortium's  purpose  is  really  two-fold.  One,  to  identify  the 
needs  of  the  cities  that  have  some  commonality.  Obviously,  a new 
solution  to  cleaning  beaches  doesn't  have  a whole  lot  of  appeal  to 
Kansas  City,  Missouri,  and  snow  removal  doesn't  have  a whole  lot  of 
appeal  to  Miami.  So,  the  first  step  really  was  to  identify  the  needs 
of  the  cities.  We  each  knew  our  own  but  we  didn't  know  those  of 
the  other  33  jurisdictions.  Questionnaires  were  sent  out,  some 
sessions  were  held,  and  I think  we  were  all  surprised  at  the  common- 
ality of  the  needs.  There  were  the  exceptions  as  the  ones  I just 
mentioned  but  most  of  the  needs  that  were  identified  by  San  Diego 
were  common  to  Atlanta,  Milwaukee,  Seattle,  San  Jose,  everybody  had 
the  same  problem--shortage  of  funds,  not  enough  people,  rising 
demand  for  public  services.  After  those  needs  were  identified,  they 
were  then  prioritized.  Those  needs  that  were  highest  in  priority 
were  then  compartmenta 1 i zed  into  ten  task  forces:  Transportation, 
Energy,  Finance  and  Personnel,  Public  Works  and  Utilities,  etc. 

Each  of  those  task  forces  then  is  represented  by  from  eight  to  ten 
of  the  3^  agencies,  and  the  task  force  then  pr i ori t i zes--wi th i n the 
needs  that  are  assigned  to  that  particular  task  force--those  eight 
to  ten  needs  that  they  wish  to  address.  The  purpose  then  is  to  find 
out  if  somebody  has  developed  the  mouse  trap  that  everybody  else 
is  looking  for  and  if  they  have,  if  we  can  identify  it  in  Seattle, 
then  we  can  transfer  it  to  other  agencies.  But,  if  none  of  the 
agencies  have  solved  the  particular  problem,  our  purpose  then  is, 
using  R6D  money,  to  either  ourselves  or  by  contract  or  in  cooperation 
with  the  Federal  labs  and  the  other  resources  that  are  available,  to 
set  to  work  to  solve  the  particular  problem.  Once  solved,  again, 
to  try  to  transfer  to  the  other  33  agencies. 

We  have  received  support  from  the  Federal  labs,  San  Diego  City  has 
participated  in  the  Science  Advisory  program,  and  A1 , Gerry,  and 
Dick  have  all  been  a help  to  us  as  Del  and  the  CIG  have  been  of  help 
to  us.  We're  just  getting  our  act  together.  We're  really  only  in 
our  third  year  of  operation.  We  are  dependent  for  funding  now,  pri- 
marily on  Federal  agencies,  but  we  think  it's  worth  doing.  We're 
committed  to  it,  the  other  cities  and  counties  are  participating, 
and  I wouldn't  be  surprised  if  some  of  you  in  this  room  won't  be 
hearing  from  us  with  a cry  for  help  somewhere  along  the  line.  We 
certainly  hope  three  or  four  years  from  now  at  a session  like  this. 


we'll  be  able  to  come  back  and  point  out  problems  that  have  been 
identified,  solved,  and  then  the  technology  transferred  to  the 
cities  of  the  country. 


A REVIEW  OF  THE  NATIONAL  SCIENCE  FOUllDATION' S INTERGOVERNMENTAL 
PROGRAMS 


Mr.  Robert  C.  Crcwford 
Direotor,  Intergovernmental  Programs 
National  Saienoe  Foundation 
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It  should  be  evident  to  you,  from  some  of  the  speakers  here  this 
afternoon  and  from  your  side  discussions  with  others  during  the 
meeting  that  there  is  a high  degree  of  enthusiasm  and  dedication 
among  the  people  who  are  involved  in  i nter^jovernmenta  1 technology 
transfer.  One  of  my  jobs  is  to  try  to  keep  up  with  this  bunch  of 
enthusiastic  rascals  that  are  stirring  up  the  system  all  over  the 
country . 

Some  of  you  might  have  noticed  an  editorial  in  Science  magazine,  in 
the  27th  of  May  issue,  which  summarized  pretty  well,  the  question  uf 
intergovernmental  science  and  technology  relations,  posing  a fairly 
succinct  statement  of  the  kinds  of  obj’ectives  that  interested  people 
are  working  toward  in  this  area.  It's  noteworthy,  1 think,  that  that 
editorial  was  signed  by  the  President's  Science  Advisor,  Dr.  Frank 
Press,  and  also  by  Governor  George  Busbee  of  Georgia,  who  happens  to 
be  the  Deputy  Chairman  of  an  intergovernmental  science,  engineering 
and  technology  advisory  panel  which  is  connected  as  part  of  the  new 
Office  of  Science  and  Technology  Policy.  That's  an  indication,  I 
think,  of  the  interest  at  the  highest  level  in  our  country  in 
exploring  what  beneficial  things  we  can  achieve  through  better  use 
of  scientific  methods  and  technologies  at  the  State  and  local  level. 
That  is  not  to  say  that  there  hasn't  been  a lot  of  technologv 
transferred  in  our  traditional  system.  We  all  know  that  that's  the 
case.  In  fact,  as  I look  around  the  room  and  at  your  agenda,  I 
see  some  Federal  agencies  here  at  this  meeting  that  have  been  in 
the  forefront  of  innovative  ways  to  try  to  maximize  the  payoff  from 
their  investments  in  the  R&D  area.  Your  group  leaders  this  morning 
are  outstanding  personalities  in  this  regard.  Programs  like  theirs 
have  made  maximum  use  of  the  available  system  we  have. 

What  we  are  talking  about  today  is  ways  of  achieving  fundamental 
changes  in  our  system  for  investing  in  R&D  and  using  R6D  outputs 
for  the  benefit  of  our  state  and  local  governments  as  they  deal  wit' 
domestic-sector  problems.  One  of  the  luxuries  that  NSF  has  in  this 
area  is  a flexibility  to  experiment  and  to  respond  to  the  state  and 
local  governments  and  the  resource-providing  institutions  of  the 
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United  States  as  they  come  up  with  new  ideas  and  new  ways  to  try  to 
deal  with  this  highly  significant  issue.  NSF  is  not  constrained 
by  a functional  mission  or  by  a traditional  bureaucratic  system. 

We  are  free  to  respond  to  state  and  local  governments,  universities. 
Federal  labs,  other  Federal  agencies,  and,  in  fact,  about  any  other 
piece  of  the  institutional  system  that  could  make  a contribution  to 
this  review  and  exploration  of  experimentation.  The  intergovernmental 
Programs  are  a part  of  the  RANN  program,  which  is  the  applied  research 
of  the  NSF.  The  RANN  program  represents  a thrust  by  the  NSF 
starting  in  about  1971  (at  the  request  of  0MB)  to  undertake  applied 
research  in  national  need  areas,  environment,  energy,  productivity, 
and,  most  recently,  in  resources.  We,  in  the  Intergovernmental 
Programs  do  not  sponsor  research,  per  se , but  we  are  fully  integrated 
with  RANN's  research  projects  in  the  sense  that  the  people  performing 
the  research  and  our  own  program  managers  in  RANN  often  use  some  of 
the  experimental  mechanisms  that  we  have  been  involved  with  around 
the  country  at  State  and  local  government  levels,  for  the  purpose 
of  facilitating  the  utilization  of  RANN's  research  products.  The 
California  Innovation  Group,  for  example,  has  been  a vehicle  for  a 
multiple  city  test  of  some  research  in  scheduling  models  that  have 
come  out  of  the  RANN  productivity  program--a  fairly  successful  test 
where  in  at  least  five  cities  the  techniques  developed  have  been 
evaluated  for  possible  application,  and  in  several  instances  applied 
in  an  operating  situation. 

The  Intergovernmental  Programs  of  NSF  are  structured  not  only  with  a 
focus  on  local  government;  they  include  State  level  activities  as  well. 
We  deal  with  State  executive  branches  with  both  the  Governors' 
offices  and  with  the  line  operating  departments  of  the  states  as 
they  try  to  maximize  the  potential  for  science  and  technology  in  their 
activities  through  different  kinds  of  organizational  relationships, 
new  communication  devices,  etc.  We  are  the  only  Federal  program  that 
we  know  of  which  has  a special  major  program  sub-element  dealing  v;i th 
state  legislatures.  We  have  worked  with  a number  of  the  legislatures 
around  the  country  in  this  often  forgotten,  but  highly  significant 
governmental  area,  to  help  them  to  improve  the  base  of  information 
on  which  they  can  make  their  policy  decisions.  With  such  improvement, 
they  are  no  longer  hampered  by  either  a total  lack  of  knowledge  or 
just  a knowledge  input  which  comes  from  perhaps  a lobby  source  which 
would  not  reflect  a total  spectrum  of  options  and  perspectives. 

We  also  have  been  involved  with,  as  a major  program  element,  and 
increasingly  so,  the  Federal  laboratory  activity  which  has  been 
described  today.  Many  of  you  are  personally  familiar  with  this  thrust. 
We  have  had  on  our  staff  since  1971  a program  manager  from  one  of  the 
participating  laboratories  to  support  the  Federal  Laboratory  Consortium, 
and  as  George  has  indicated,  we  have  been  able  to  provide  some  small 
amount  of  program  funds  to  help  facilitate  the  testing  of  ways  by 
which  the  Federal  Laboratories  may  heighten  their  assistance  to  State 
and  local  governments. 
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I would  like  to  elaborate  to  some  extent  on  the  degree  of  interest 
in  the  subject  of  Intergovernmental  science  relations  which  goes  far 
beyond  the  NSF  and  even  far  beyond  individual  cities  and  states 
which  are  involved  in  projects.  For  example,  I came  to  California 
by  way  of  Tucson  where  there  is  underway  a national  meeting  of  the 
U.  S.  Conference  of  Mayors.  One  of  the  reasons  that  I stopped  at 
this  major  national  meeting  is  that  one  of  the  activities  underway 
was  the  first  meeting  of  the  U.  S.  Conference  of  Mayors  Committee 
on  Science  and  Technology.  This  Committee  is  composed  of  repre- 
sentative mayors  of  cities  large  and  small  all  over  the  country. 

These  mayors  came  together  to  develop  policies  and  directions  for 
the  Conference  with  regard  to  the  question  of  how  technology  can 
be  made  as  relevant  as  possible  to  help  local  officials  deal  with 
issues  they  face  in  their  towns.  This  committee  met  and  proposed 
a resolution  which  expected  to  be  passed  at  their  plenary  session 
this  morning.  The  title  of  this  resolution  is  "Priority  Concerns 
Related  to  National  Science  and  Technology  Efforts."  The  resolution 
focuses  on  the  activities  of  the  Office  of  Science  and  Technology 
Policy,  the  Office  of  Management  and  Budget,  and  local  governments 
from  the  standpoint  of  how  these  institutions  can  facilitate 
technology  transfer  on  the  domestic  scene,  and  it  includes  a special 
section  on  the  Federal  laboratories.  This  section  calls  for  a 
continuation  of  efforts  to  organize  the  labs  to  heighten  their  impact 
on  technology  transfer  to  the  State  and  cities.  The  resolution 
calls  upon  the  Carter  Administration,  and  I quote,  "to  take  whatever 
steps  that  are  currently  available  under  present  law  to  direct  this 
resource  (the  labs)  away  from  its  almost  exclusive  dedication  to  the 
purposes  of  the  central  government  and  to  those  of  state  and  local 
governments.  We  call  upon  the  Congress  to  examine  this  resource 
and  to  lift  any  restrictions  which  might  limit  the  realization  of 
these  potentials."  The  resolution  also  talks  about  strengthening 
local  institutions  to  permit  them  to  better  participate,  restructuring 
local  government  where  necessary  to  facilitate  this.  It  is  inter- 
esting to  note  at  the  bottom  of  the  resolution  where  the  Conference 
has  a practice  of  showing  the  projected  cost  of  implementing  this 
resolution  that  there  is  estimated  to  be  no  additional  costs  required. 
This  is  in  contrast  to  many  of  their  resolutions  concerning  reverse 
economic  stimulation,  etc.,  wherein  the  estimated  cost  is  stated  in 
billions  of  dollars.  The  Conference,  in  this  case,  feels  that  to  do 
this  reflects  basically  a redirection  of  and  maximizing  the  impact 
of  already  in-place  investments. 

As  another  example  of  national  interest  in  State  and  local  technology 
transfer,  the  National  Conference  of  State  Legislatures,  which 
represents  the  7,600  state  legislators  all  over  the  country,  formed 
both  a Committee  on  Science  and  Technology  and  a Committee  on 
Information  Needs.  This  latter  committee  focuses  to  a large  extent 
on  the  issue  of  how  they  can  improve  the  data  inputs  that  they 
receive  on  which  they  base  their  decisions  regarding  environmental 
matters,  energy,  growth,  etc.  One  of  the  features  of  a project 
that  the  NCSL  is  proposing  is  to  tie,  for  the  first  time,  selected 
State  legislatures  to  the  Congressional  Research  Service  of  the 
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Library  of  Congress.  This  proposed  demonstration  project  would  gain 
access  on  an  on-line  basis  into  the  computer  of  CRS  so  that  the 
States  may  obtain  information  about  various  technology-related 
problems,  the  list  of  advisors  and  technical  specialists  around  the 
country  that  the  Congress  uses  in  various  areas,  etc.  This  type  of 
activity  is  indicative  of  the  sorts  of  potential  that  the  State 
legislatures  see  in  increased  linkages  to  S£T  resources.  The  NCSL's 
interest  and  that  of  the  National  Governors  Conference,  as  well  as 
the  interest  of  certain  parties  in  the  Congress  led  to  the  authori- 
zation, this  year,  of  a $3  million  program  that  is  called,  "The 
State  Science  and  Engineering  Technology  Program  (SSET)"  under  which 
there  will  be  grants  made  available  to  each  of  the  State  executive 
branches  and  the  State  legislatures  up  to  the  amount  of  $25,000 
with  matching  requirements  of  $12,500  (if  they  get  $25,000),  to 
help  them  conduct  studies  of  their  posture  and  capabilities  in 
science  and  technology  and  how  they  might  better  organize  themselves 
to  take  advantage  of  the  resources  that  exist  in  their  states. 
Presumably,  those  resources  could  include  the  universities,  the 
private  non-profits,  and  private  profit  institutions  that  might  help 
with  technologically  related  matters,  and  of  course,  it  could 
involve  the  Federal  laboratories  as  well.  There  are  indications  that 
the  states,  as  they  prepare  to  respond  to  this  overture  by  Congress, 
will  probably,  in  some  instances  at  least,  involve  Federal  labs  in 
their  proposals  in  some  way.  First-round  proposal  for  this  program 
will  have  to  be  submitted  to  the  Foundation  by  August  15,  hopefully 
to  be  funded  this  fiscal  year. 

Our  final  element  that  must  be  mentioned  is  the  Intergovernmental 
Science  Technology  and  Engineering  Panel,  which  is  part  of  OSTP. 

That  panel  is  a statutory  panel,  and  was  included  in  the  science 
priority  act  that  George  Linsteadt  referenced  in  his  remarks.  It's 
composed  of  four  governors,  four  mayors,  a number  of  city  managers, 
a number  of  county  officials,  three  or  four  state  legislators,  and 
has  staff  to  support  it  in  the  Office  of  Science  and  Technology 
Policy.  Governor  Busby,  the  Panel's  Vice  Chairman,  has  indicated 
that  if  the  Panel  isn't  for  real,  with  some  quantifiable  impact,  he 
does  not  want  to  be  involved.  The  thrust  of  the  Panel  is  to  make 
meaningful  inputs  into  the  Federal  government's  policy  in  the  science 
and  technology  area.  I have  a document  here  which  reports  on  the 
activities  of  that  panel  for  one  recent  month.  The  kind  of  sound  and 
fury  that  is  coming  out  of  that  Panel,  and  the  fantastic  high  level 
of  activity,  bodes  well  for  achievement.  The  state  and  local 
government  people  and  the  science  and  technology  policy  people  that 
are  involved  with  the  Panel  are  quite  serious  about  it  and  dedicated, 
and  are  driving  to  make  it  an  effective  instrument  for  impacting 
on  national  pol icy. 

It  is  clear,  I think,  that  all  of  us  involved  in  this  effort  in  the 
Foundation  and  the  other  Federal  agencies  with  whom  we  relate,  the 
people  and  projects  in  State  and  local  government  with  whom  we 
deal,  and  those  persons  in  the  technology  resource  community  who 
have  gotten  involved  in  the  State  and  Local  government  service 
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Dr.  Kay  Inaba 

President,  XYZYX  Information  Corporation 


II  will  not  be  so  presumptuous  as  to  try  to  summarize  this  afternoon's 

session  for  the  total  audience.  However,  I thought  it  might  be  of 
some  value  to  summarize  the  session  from  a practitioner's  point  of 
I view. 

I I am  now  in  business  as  a practitioner.  I used  to  be  a researcher, 

like  many  of  you.  However,  I am  now  a practitioner  in  business,  and 
to  stay  in  business  I must  make  good  use  of  research  results.  Thus, 

1 (and  fellow  practitioners)  are  your  primary  customers. 

. 

I believe  that  three  basic  messages  were  conveyed  today.  Based  on 
these  messages,  both  individual  conclusions  as  well  as  a general 
conclusion  are  in  order. 

1.  The  successful  research  utilization  discussed  today  result 
primarily  from  practitioners  who  are  sympathetic  to  or  have 
been  members  of  the  research  community.  Mr.  Gerald  Miller 
is  a prime  example  of  such  a practitioner. 

It  would  appear  that  one  answer  to  the  research  utilization 
problem  would  be  to  xerox  Mr.  Miller.  Obviously,  this  can't 
be  done  physically,  but  is  possible  conceptually.  That  is, 
we  should  train  more  pract i t i oners . 

Consider  the  normal  practitionei — or  our  cohorts  in  the 
appl ication  of  personne 1 -re lated  research  data.  One  such 
practitioner  is  an  engineering  drop-out  shunted  to  a position 
as  a training  director.  This  is  usually  an  individual  with 
little  or  no  knowledge  of  the  learning  process--but  controls 
millions  of  dollars  in  training  equipment  and  programs. 

Another  colleague  is  the  director  of  publications  with  a 
technical  writing  background.  He  has  lived  in  a world  of 
specifications  with  little  or  no  concern  with  usability.  Or, 
take  the  case  of  the  design  engineer  taught  from  school  onward 
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only  about  equi pment--and  virtually  nothing  about  the  people 
who  operate  or  maintain  the  equipment. 

Too  often,  we  put  the  pressure  on  the  researcher  to  improve  the 
ut'  ization  of  his  research  results.  I believe  the  focus  is 
on  ^.le  wrong  place.  Most  good  researchers  are  neither  inclined 
towards  nor  skilled  at  applications.  Why  force  him  into  a level 
of  incompetence. 

The  appl i ed  researcher  must  listen  to  and  meet  the  needs  of 
the  practitioner.  However,  I believe  that  utilization  should 
be  the  task  of  the  practitioners.  I believe  more  emphasis 
and  effort  should  be  directed  to  developing  practitioners, 
with  an  effective  line  of  communication  with  the  research 
commun i ty . 

Within  DoD,  the  practitioners  should  be  freed  from  artificial 
criteria,  such  as  "publish  or  perish."  The  practitioner  should 
be  rewarded  for  successes  in  applications — not  research. 

2.  An  expenditure  of  $26  billion  for  research  (50^  in  DoD)  would 
inoicate  that  it  is  not  enough  to  just  try  to  improve  research 
utilization.  We  (researchers  and  practitioners)  must 
succeed . 

I suspect  that  the  research  community  no  longer  has  the  luxury 
of  just  trying.  The  pressure  from  politicians,  accountants, 
bureaucrats,  etc.,  experienced  to  date  can  probably  be 
expected  to  i ncrease , unless  utilization  is  improved. 

Unfortunately,  the  pressure  is  forcing  researchers  to  act  as 
pract i t ioners--of ten  with  less  than  satisfactory  results. 

1 believe  that  problems  of  application  are  bona  fide  subjects 
for  research.  NPRDC's  study  of  the  role  of  the  change  agent 
in  introducing  new  equipment  items  is  a step  in  the  right 
direction.  More  such  research  and  attention  are  needed. 

I personally  believe  that  more  attention  should  be  given  to 
the  following  aspects  of  technology  transfer; 

a.  Transition  from  research  to  development. 

In  the  human  factors  field,  concepts  seem  to  jump  from 
the  research  state  to  the  application  state.  Frequently, 
this  results  in  technology  either  staying  in  the  research 
mode  too  long,  or  premature  application. 

For  example,  the  Job  Performance  Aid  concept  has  been 
"studied  to  death"  during  the  past  seven  years.  Yet, 
the  state-of-the-art  of  JPAs  has  not  been  advanced 
signi f icantly. 
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In  contrast,  task  analysis  techniques  have  been  imposed  on 
industry  without  adequate  research.  Consequently,  much 
data  have  been  created  over  relatively  unimportant  events. 


"I 


b.  More  systems  approach. 

We  need  more  human  factors  people  to  apply  the  systems 
approach.  Personnel-related  concepts  and  techniques  need 
to  be  integrated  into  a system  for  proper  application. 

Often  such  applications  are  assigned  to  engineers  because 
of  their  systems  experience.  Unfortunately,  they  too 
' often  do  not  understand  the  human  factors  variables. 

Consequently,  the  effect  of  the  new  is  often  dissipated, 

I or  cannot  be  identified. 

c.  Procurement  practices. 

More  professionals  in  people-related  research/applications 
ajree  that  the  competitive  bid  approach  is  not  effective. 
Yet,  few  have  dared  to  do  much  about  the  problem. 

Generally,  ASPR  has  considerably  more  flexibility  than 
most  researchers  assume.  Unfortunately,  many  researchers 
are  intimidated  by  the  procurement  agents  and  take  the 
path  of  least  "risk." 

However,  it  would  probably  be  worthwhile  to  examine  ASPR 
to  see  if  it  meets  the  needs  of  personnel-related  research/ 
applications.  The  ASPR  is  supposed  to  serve,  rather  than 
[ be  the  master. 

j 3.  The  consortium  works.  The  frequent  reference  to  the  consortium 

indicates  that  the  consortium  works.  In  fact,  the  speakers 
were  all  saying  that  the  research  results  are  sorely  needed  to 
help  the  local  government. 

I During  the  past  few  years,  peop 1 e- re  1 ated  research  has  been  the 

I target  of  much  cri t ici sm--for  inadequate  research  utilization. 

I Thus,  it  is  encouraging  to  hear  practitioners  say  they  are 

1^  utilizing  the  research  results  and  services. 

[ Personally,  I believe  that  considerable  advancements  are  being 

; made  by  personne I -re  1 ated  researchers.  In  fact,  the  value  has 

1 probably  been  far  greater  than  given  credit  by  the  GAO.  I 

suspect  that  much  of  the  problem  is  inadequate  communication 
with  the  lay  evaluators  and  managers.  Perhaps  the  practitioners 
may  be  of  some  help. 

A truce  is  needed--between  researchers  and  practitioners. 

The  small  group  of  practitioners  tend  to  resent  researchers 
telling  them  how  to  appIy--often  using  impractical  criteria. 
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In  turn,  researchers  are  wary  of  practitioners  who  seem  to 
be  salesmen. 

I believe  it's  time  both  parts  of  the  community  recognize 
that  we  need  each  other,  and  call  a truce.  We  need  to  sit  at 
a truce  table  and  determine  our  mutual  needs,  and  develop 
an  implementation  program. 

I would  suggest  the  above  be  done  by  a working  committee 
of  people  from  the  following  areas: 

Practi tioners 
Research  Centers 
Academi a 

Procurement  Agencies 
Upper  DoD  management 

This  group  should  develop  a one,  five,  and  a ten-year  program 
of  appl i cations . 1 suspect  the  program  wi 1 1 include  training 

of  practitioners,  communication  with  the  research  community, 
defining  research  needs,  identification  of  key  development 
programs  and  development  of  spokespersons. 
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CASE  PR0BLE1  ANALYSIS 


SUMMARY 


In  this  session  the  conferees  met  in  smalt  groups  to  analyze  case 
problems  of  RDT&E  project  outcomes  that  have  not  been  optimally 
utilized  and  to  formulate  strategies  for  improving  utilization. 

The  small  group  discussions  were  then  summarized  by  the  group  lead- 
ers in  plenary  session.  The  case  problems  (contained  in  Appendix  E) 
were : 

Stereoscopic  Viewing 
REALTRAIN 

Naval  Training  School 
Armor  School 

Armed  Services  Vocational  Aptitude  Battery 

Bert  Griffis,  Rehabilitation  Services  Administration,  reported  his 
group’s  discussion  of  the  stereoscopic  viewing  case  problem.  He 
emphasized  the  need  for  the  researcher  to  understand  the  total  sys- 
tem context,  including  such  factors  as  vested  interests.  He  also 
pointed  out  that  transition  to  the  operational  community  might  have 
been  successful  if  a linking  agent  had  been  assigned  at  the  beginning 
of  the  project. 

Dr.  Herbert  Clark,  Air  Force  Human  Resources  Laboratory,  also  dis- 
cussed the  stereoscopic  viewing  case.  He  stressed  the  need  for 
closer  coordination  with  the  customer  and  for  a formal  agreement 
as  to  what  criteria  will  determine  whether  a given  result  will  be 
used.  He  also  noted  that  there  is  more  than  one  potential  user,  so 
there  is  a need  to  determine  where  else  the  results  might  be  applied. 

Dr.  Ronald  Havelock,  University  of  Michigan,  reported  on  the  REALTRAIN 
case  problem.  He  discussed  a number  of  issues  suggested  by  this  case, 
including : 

. The  problem  of  premature  adoption. 

. The  problem  of  the  self-fulfilling  prophecy,  whereby  defenders 
of  the  status  quo  engage  in  sloppy  implementation  of  the  new 
technique. 

. The  importance  of  stringent  adoption  of  some  innovations. 

. The  fact  that  some  innovations  get  to  be  so  technically  elegant 
that  the  probability  of  breakdown  is  increased. 

. The  question  of  who  should  control  the  adoption  decision. 

Dr.  William  McClelland,  HumRRO,  discussed  the  naval  training  school 
case  problem.  He  pointed  out  the  importance  of  determining  the  com- 
parability of  sites  that  may  be  candidates  for  the  installation  of  a 
particular  innovation.  He  also  noted  that  this  case  dramatically 
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illustiHxtes  the  need  for  aost-effeotiveness  analysis,  and  for  adequa.te 
provision  for  selling  promising  research  results. 


Dr,  Meredith  Crawford,  George  Washington  University,  also  reviewed 
the  naval  training  school  case  and  stressed  the  need  for  agreement 
between  operator  and  researcher  as  to  haw  proficiency  should  be  mea- 
sured and  whether  performance  in  a training  situation  reflects  the 
actual  requirements  of  the  job.  He  also  emphasized  the  importance 
of  cost-effectiveness  analysis,  and  the  fact  that  findings  don't  get 
implemented  simply  by  inscribing  them  in  technical  reports. 

Dr.  Arthur  Drucker,  Army  Research  Institute,  reported  his  group  's 
analysis  of  the  naval  training  school  and  stereoscopic  viewing  cases. 
He  commented  on: 

. The  need  for  user-researcher  agreements  and  commitments  at  the 
inception  of  the  research. 

. The  need  for  "front-end"  analysis,  including  analysis  of  i 
quirements,  cost  benefits,  risks  and  alternate  solutions, 

. The  need  for  sensitivity  to  the  total  system  implications  of 
a particular  innovation. 
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CASE  prqble;^  analysis 


CASE  PROBLEM  DISCUSSION 


DR.  GLASER;  We'd  like  to  hear  from  the  spokesmen  for  each  of  the  groups 
now  with  regard  to  their  analysis  of  the  prepared  cases.  Before  we 
do  so  I'd  like  to  share  an  observation  from  my  visits  to  the  several 
groups  in  which  the  cases  were  being  discussed.  One  of  the  things 
that  struck  me  was  a reaffirmation  of  an  experience  most  of  us  have 
had  to  the  effect  that  even  when  you  get  very  knowledgeable  people 
together  in  any  given  field  to  work  on  what  might  best  be  done  to  deal 
with  a given  problem,  we  find  a good  deal  of  honest  difference  of 
opinion.  These  differences  often  can  be  enriching  and  rewarding 
if  they  can  be  integrated  constructively. 

1 was  thinking  that  after  Mrs.  Ford  and  Mrs.  Rockefeller  had  their 
mastectomies  and  that  type  of  problem  got  headline  attention,  the 
Today  Show  had  a five-day  panel  of  the  leading  surgeons  in  the 
country  to  discuss  the  state-of-the-art  in  treatment  of  mastectomy. 

The  panel  included  Dr.  Crile  of  Cleveland  Clinic  and  others  who  are 
very  well  known,  such  as  a top  surgeon  from  Sloan  Kettering  in  New 
York.  The  difference  of  opinion  among  these  eminent  specialists 
tended  to  leave  the  potential  consumer,  so  to  speak,  in  a quandary. 

And  yet  everybody  had  something  to  say  that  needed  to  be  taken  into 
account.  It  wasn't  until  Friday  that  the  other  panelists  seemed  to 
understand  that  Dr.  Crile,  who  advocated  excising  just  the  tumor 
in  many  cases,  was  not  saying  that  this  was  the  preferred  treatment 
under  all  conditions.  He  was  saying  that  if  the  lymph  nodes  were 
not  affected,  the  evidence  according  to  his  longitudinal  study  was 
that  the  recovery  rate  was  Just  as  good  from  the  simpler  operation 
as  compared  to  the  radical  surgery.  Where  the  lymph  nodes  were 
affected,  he  too  would  opt  for  a radical  mastectomy.  But  the  fact 
that  they  started  out  with  a different  orientation  led  to  quite  a 
few  days  of  difference  before  the  panelists  realized  that  they  were 
not  as  far  apart  as  they  seemed. 

Well,  in  terms  of  what  to  do  about  some  of  these  case  problems, 
every  comment  I heard  was  intelligent  and  relevant,  but  often  re- 
flected different  views  as  to  what  was  of  primary  importance.  That 
was  one  of  the  reasons  for  planning  a segment  of  this  workshop  to 
deal  with  actual  cases  involving  utilization  difficulties.  After 
all  of  these  theory  sessions  in  the  last  two  days,  the  thought  was 
to  give  you  a chance  to  grapple  in  a concrete  way  with  problem 
s i tuat i ons . 


*The  case  problems  discussed  in  this  session  are  contained  in 
Append i x E . 
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Now,  may  we  hear  from  the  spokesman  for  Group  §] . Would  you  tell 
us  which  of  the  several  cases  you  discussed,  and  what  your  recommenda- 
tions, findings  or  questions  were. 


BERT  GRIFFIS:  (Spokesman  for  Group  ^l)  We  chose  the  case  problem  that 
concerned  stereoscopic  viewing.  The  fact  that  it  wasn't  used  may 
mean  that  there  were  some  powerful  factors  lurking  in  the  background 
to  prevent  that,  and  perhaps  some  of  them  were  vested  interests, 
such  as  the  pilots  who  like  to  fly  more  to  take  pictures  and  even 
the  people  who  develop  the  negatives.  Someone  made  the  excellent 
point  that  people  who  had  been  doing  this  type  of  research 
should  have  had  a systems  analysis  point  of  view,  they  should  have 
had  a picture  of  the  whole  system,  and  if  they  had,  they  would  have 
known  about  all  of  these  other  factors  that  needed  to  be  taken  into 
account.  We  felt  we  didn't  know  quite  as  much  about  the  research 
itself  as  we  might  have.  Conceivably,  if  the  procedure  had  been 
rigorously  analyzed  it  might  not  have  been  as  valid  as  presented. 
Another  point  made  was  that  if  the  students  were  better  motivated 
to  go  through  the  training  when  the  stereoscopic  slides  were  being 
used,  greater  benefit  from  the  training  might  have  outweighed  the 
greater  cost  of  the  stereoscopic  approach.  One  member  of  the  group 
made  the  point  that  since  it  appeared  to  be  such  a clear-cut  case, 
that  is  the  non-stereoscopic  data  were  just  as  good  and  cost  a lot 
less,  that  he  would  have  implemented  that  by  fiat  on  the  basis  of 
cost-benefit.  But  another  person  made  the  point  that  the  people 
who  have  the  power  to  implement  it  by  fiat  might  not  have  been  sold 
on  doing  so. 

DR.  GLASER:  Might  some  other  members  of  that  group  wish  to  add 
anyth i ng? 

DR.  HAVELOCK:  I have  a suggestion.  If  we're  talking  about  one  case, 
wouldn't  it  make  sense  for  the  various  people  who  worked  on  that 
case  all  to  make  their  presentations  at  the  same  time? 

DR.  GLASER:  Yes,  I'm  planning  to  do  that.  But  first,  in  terms  of 
the  other  members  of  the  group  who  participated  in  this  discussion, 
does  anyone  wish  to  add  additional  points? 

BERT  GRIFFIS:  There  were  some  inputs  that  I omitted. 

DR.  GLASER:  Go  ahead. 

BERT  GRIFFIS:  1 think  the  point  was  made  that  if  there  had  been 
somebody  assigned  at  the  beginning  of  that  effort  as  a sort  of 
technology  transfer  agent  of  a liaison  with  the  operational  community, 
and  that  party  or  parties  had  sat  down  with  the  intended  users  and 
gotten  them  ready  to  accept  this,  the  transition  might  actually  have 
been  successful.  At  the  very  least  you  might  not  have  gone  to  the 
trouble  of  doing  all  that  research  if  you  learned  there  wasn't  a 
ready  and  willing  climate  of  acceptance  for  it.  But  the  case  was 
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unclear  as  presented  as  to  whether  anything  had  been  done  to  pave  the 
way  for  that  transition,  and  it  was  also  unclear  as  to  whether  there 
had  been  a meeting,  formal  or  informal,  between  the  users  and  the 
researchers  to  find  out  why  in  the  world  this  thing  was  not  being 
accepted.  They  must  have  had  some  reasons,  good  or  bad,  and  it 
would  be  very  instructive  to  have  learned  what  they  were  and  try  to 
deal  with  them. 

DR.  GLASER:  Did  any  other  group  take  up  this  problem?  I thought 
we  might  feel  free  to  invite  questions  from  the  audience  about  this 
but  if  another  group  took  up  the  case,  we  probably  should  hear  from 
that  group  next.  Further,  before  we  leave  this  case  perhaps 
Jay  Uhlaner  or  Art  Drucker  or  whoever  else  in  the  Army  had  cognizance  of 
this  could  tell  us  in  a little  more  detail  what  really  happened. 

BERT  GRIFFIS:  I have  one  other  point,  which  was  made  by  someone 
else  too.  The  visual  pleasure  of  seeing  those  two-dimensional  slides 
might  have  been  a factor  in  their  clinging  to  it.  Many  of  the  people 
working  with  it  might  have  felt  that  the  non-stereoscopi c pictures 
were  more  difficult  to  use  and  to  get  the  data  out  that  they  needed. 

A MEMBER  OF  THE  AUDIENCE:  I have  to  comment  on  that.  The  visual 
pleasure  of  the  photographic  interpreter  seems  irrelevant.  I just 
can't  relate  that  concern  at  all  to  the  fact  that  twice  as  many 
planes  have  to  be  flown,  with  twice  as  many  chances  to  get  pilots 
killed.  I don't  believe  that  the  pilots  who  had  to  fly  that  extra 
mission  and  might  get  shot  down  would  put  a hell  of  a lot  of  weight 
on  the  trivial  consideration  of  the  interpreter's  having  more  visual 
p 1 easu  re . 

I would  also  like  to  comment  that  we  didn't  have  the  facts  as  to  who 
comprised  the  experimental  group.  If  in  fact  they  used  untrained 
students  as  opposed  to  trained  photo- i nterpreters , the  probability 
is  that  it  would  be  easier  to  train  the  experimental  group  using  the 
stereoscopic  approach  as  opposed  to  training  photo- i nterpreters  using 
two-dimensional  photographs.  I think  that  this  point  has  to  be  known 
before  the  case  can  be  analyzed  adequately. 

DR.  DRUCKER:  I can  supply  you  with  the  factual  information  on  that; 
both  groups  used  trained  photo- i nterpreters . 

DR.  GLASER:  Art,  did  you  wish  to  add  anything  to  this? 

DR.  DRUCKER:  Well,  no,  not  in  the  context  we've  been  discussing. 

DR.  GLASER:  Well  then,  can  we  hear  from  the  other  group  that  tackled 
this  case? 

DR.  CLARK:  I think  we  picked  this  case  because  I was  interested  in 
it.  I heard  about  it  some  time  ago  when  the  study  was  being  conducted 
and  I didn't  know  what  the  results  were.  I think  that  a number  of 
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good  points  have  been  made  thus  far  with  which  I would  agree,  with 
the  exception  of  one.  The  one  was  the  statement  that  the  results 
weren't  used  because  they  didn't  lead  to  a specific  change,  such 
as  a change  in  policy.  The  fact  of  the  matter  is,  I think  the 
results  were  used  in  making  a decision.  But  what  might  first  appear 
to  be  a clear-cut  decision  in  terms  of  cost-effectiveness  really 
isn't  so  c 1 ea r-cut . 

I think  the  point  already  made  about  the  pilots  was  very  well  taken. 

It  certainly  wouldn't  be  worthwhile  to  fly  a larger  number  of  combat 
missions  to  get  the  additional  images  just  for  the  satisfaction  of 
the  photographic  worker,  if  there  is  that  danger  involved.  But 
what  could  have  been  done  at  the  outset  of  this  study?  We  would 
agree  with  what  was  already  said,  that  there  should  perhaps  have 
been  more  coordination  with  the  customer  and  a formal  agreement 
made  as  to  what  sorts  of  results  were  to  be  provided,  and  what 
criteria  would  be  used  to  determine  what  it  would  take  to  convince 
the  user  (decision-maker)  to  go  one  way  or  the  other.  For  example, 
it  might  be  stipulated  that  the  critical  factor  in  the  decision-making 
hinges  on  whether  one  procedure  is  a certain  percentage  better  than 
the  other  in  terms  of  number  of  targets  detected,  and  if  so,  vie  would 
adopt  that  procedure. 

In  this  case,  perhaps  the  researcher  was  trying  to  impose  an  idea 
or  finding  on  the  customer  rather  than  the  customer  approaching 
the  researcher  with  a problem,  which  makes  it  a more  difficult 
situation.  But  what  if  we  did  get  with  the  customer  and  tried  to 
make  a formal  agreement  and  he  said,  "No,  we  don't  want  you  to  do 
the  research."  We  addressed  the  question  of  whether  we  should  do 
it  anyway.  I would  be  rather  interested  in  other  points  of  view  on 
that . 

Further,  I thought  a very  important  point  brought  up  by  one  of  the 
members  of  the  group  was  that  there's  more  than  or.e  user.  It's  not 
just  the  fleet;  it's  not  just  the  operational  command;  we  also  have 
a responsibility  to  potential  users  in  the  RSD  community.  That  is, 
if  the  results  of  this  project  have  scientific  merit;  if  the  results 
are  general izable  and  not  just  something  that  would  pertain  only 
to  a specific  operational  setting;  if  it's  something  that  is  pertinent 
to  the  operational  community,  then  we  have  the  responsibility  to 
report  it  to  the  scientific  community  and  retain  it  in  a file  for 
eventual  use.  What  actually  happened?  The  results  were  not  used 
to  the  extent  that  the  researcher  might  have  liked  them  to  have  been 
used. 

What  else  might  have  been  done?  My  recommendation  here  was  to  say, 
"Find  out  where  else  the  results  might  be  applied."  We  could  put 
the  results  in  the  category  of  potential  use  and  report  this  to  the 
scientific  community.  We  did  talk  to  some  extent  about  how  far  we 
should  go  in  marketing  the  results  or  in  trying  to  convince  the 
user.  I don't  think  we  should  chase  the  potential  user  to  the 
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degree  that  it  is  taking  too  much  away  from  our  RED  time.  For 
myself,  if  I ask  a researcher  to  try  to  get  these  results  imple- 
mented, I say,  "You've  got  about  a month  to  do  it;  after  that,  get 
back  to  doing  RED."  We  in  research  can't  chase  the  potential  users 
forever,  but  you  do  the  best  you  can.  Also,  times  do  change.  (I 
think  this  study  was  done  some  time  ago).  For  example,  a previous 
speaker  alluded  to  the  fact  that  there  were  great  cost-benefits 
associated  with  this  study.  There  were,  and  cost-benefit  is  a more 
important  consideration  today  perhaps  then  it  was  when  this  study 
was  done.  It  might  be  appropriate  to  brief  this  study  again-- 
to  take  it  down  the  road  and  discuss  it  further.  We  found  it  an 
interesting  problem  in  research  utilization.  I wish  I could  say 
we  have  some  concrete  recommendations  as  to  what  can  be  done,  but 
basically  we  think  you  should  get  with  the  customer  in  the  first 
place  and  try  to  come  up  with  an  agreement.  If  you  can't  reach 
an  agreement,  perhaps  you  should  proceed  anyway  if  it  has  scientific 
merit.  But  if  you  do  come  up  with  an  agreement,  I think  you  should 
establish  criteria  as  to  what  the  customer's  responsibility  is  when 
you  give  him  the  results;  and  what  it  would  take  from  us  to  convince 
him  that  this  should  be  implemented.  I think  this  pertains  to  a 
number  of  cases.  I probably  left  out  a lot  of  the  important  comments 
that  were  brought  up  in  the  meeting.  Dr.  Bryan,  you  certainly 
had  some  good  points,  would  you  like  to  elaborate  on  them?  Or 
anybody  else  in  the  group? 

DR.  BRYAN:  No,  I think  my  points  have  been  brought  out. 

DR.  GLASER:  It  might  be  well  to  clarify  a point  of  information  about 
the  facts  of  this  case.  What  was  standard  practice,  the  stereoscopic 
approach  or  the  other  one,  or  possibly  some  alternative? 

DR.  UHLANER:  I will  be  in  a better  position  to  make  a comment  after 
this  discussion,  because  otherv/ise  I think  it  kills  the  purpose  of 
the  discussion.  But  that  one  I can  answer,  standard  practice  was 
definitely  stereoscopic. 

DR.  GLASER:  Then,  stereoscopic  was  standard  practice  and  the  other 
was  an  innovation,  which  from  the  research  point  of  view  turned  out 
to  be  more  cost-effective  but  didn't  result  in  any  decision  to 
change.  One  other  question;  were  all  the  decision-makers  who 
would  have  something  to  say  about  which  of  the  two  methods  would 
be  used  involved  in  this  study?  Did  they  want  this  study?  In 

other  words,  who  was  the  sponsor?  Should  we  save  that  for  later? 

Does  anyone  else  want  to  comment  here? 

DR.  UHLANER;  Yes,  I just  want  to  clarify  something  that  Jim  Clark 
said.  He  said,  and  I believe  the  small  group  was  going  along  with 
it,  that  if  this  study  had  merit  and  there  was  no  user  out  there,  the 

researcher  ought  to  go  ahead  and  do  it  anyway.  I would  disagree 

with  that  to  a considerable  extent,  I guess,  as  it  would  really 
hinge  on  the  interpretation  of  who  is  a user  and  who  is  a producer. 
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il  would  say  that  the  R&D  community  is  a user,  and  the  researcher  ought 

to  go  ahead  and  do  it  anyway  rf  the  RSD  community  is  willing  to  fund 
it  and  put  it  in  as  part  of  their  program.  However.  I don't  think  the 
t research  ought  to  be  undertaken  with  the  idea  that  if  the  researcher 

r scouts  around  and  can't  find  a user  but  nevertheless  feels  pretty 

[ sure  that  it's  a good  idea,  he  should  go  ahead  and  do  it  anyway. 

- Under  all  circumstances  it  seems  to  me  that  what  you're  seeking  is 

another  appropriate  user,  and  the  people  in  the  R£D  community  con- 
stitute a class  of  users.  You  may  well  find  research  that  needs  to 
be  done  because  it  will  contribute  to  the  technology  base.  This 
study,  or  research  project,  was  entirely  responsive  to  R&D  and  for 
that  matter  at  that  time  also  to  some  inter-service  user  requirements, 
but  the  user  was  the  R&D  community.  The  users  were  the  very  people 
involved  in  the  actual  intelligence  evaluation,  so  it  was  very  fully 
staf fed--wh i ch  was  a plus,  and  the  design  was  good,  and  the  research 
was  done  properly.  The  minus  was  that  there  was  indeed  another  class 
of  interested  parties--!  don't  know  if  I would  call  them  users--and 
that  was  American  industry,  who  were  spending  lots  and  lots  of  money 
building  these  aerial  surveillance  systems.  In  fact,  the  director  of 
the  project  gave  a very  sophisticated  and  extensive  briefing  to  American 
industry  that  was  brought  together  for  this  subject.  About  six  months 
after  that  briefing  it  came  to  my  attention  that  there  were  recommen- 
dations (from  industry)  that  the  project  director  should  be  fired 
because  in  some  way  everything  he  reported  was  di scredi ted--the  data, 
the  approach,  and  so  rn.  It  became  very  clear  that  certain  elements 
of  industry  stood  to  lose  a great  deal  of  nnoney  if  we  opted  for  the 
nonstereo  system.  Industry  argued,  for  example,  that  you  absolutely 
can't  measure  heights  of  mountains  unless  you  have  stereo  views.  Well, 
how  many  times  do  you  measure  heights  of  mountains?  We  pointed  out 
that  these  were  tactical  data;  that  we  were  not  trying  to  produce 
relief  maps;  that  the  commanders  were  satisfied.  Yet,  the  final 
decision-making  inputs  in  this  instance  were  largely  in  the  hands 
of  American  industry  and  I think  that  sort  of  puts  the  problem  in 
perspective.  These  are  the  kinds  of  sophisticated  and  subtle  points 
that  we  sometimes  aren't  aware  of  when  we  take  a look  at  the  straight 
interpretation  of  research  data. 

DR.  GLASER:  Are  there  any  other  comments  on  this? 

RESPONSE:  Yes,  I think  Dr.  Uhlaner  may  have  missed  two  tactics  that 
might  have  changed  what  he  has  described.  One  would  have  been  to 
inform  Jacf  Anderson,  and  the  other  to  inform  Senator  Proxmire. 

DR.  GLASER:  Are  there  any  other  questions  or  comments  on  this  case? 

DR.  UHLANER:  I don't  think  that  last  comment  was  trivial--!  think 
there  might  be  some  cases  where  what  I have  described  is  an  "anti- 
cipatable"  type  of  problem,  and  it's  no  news  that  there  was  a lot  of 
industry  money  behind  this,  so  when  you're  thinking  of  utilization 
of  R&D  results  you  have  to  think  about  social  system  factors  that 
could  lead  to  the  implementation  of  your  results.  We  were  aware  of 
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that,  and  we  felt  that  rather  than  asking  permission  from  American 
industry  to  run  objective  studies  on  the  comparative  value  of  the 
two  systems,  we  had  an  obligation  to  the  taxpayer  and  the  Defense 
Department . 

QUESTION:  Granted  we  have  such  an  obligation.  But  to  what  extent 
did  you  involve  yourselves  in,  say,  mass  media  or  communication  that 
would  reveal  these  special  interest  pressures? 

DR.  UHLANER;  Oh,  no;  the  thing  is  that  you  would  lose. 

QUESTION;  In  other  words,  in  effect  you're  saying  that  you  are 
willing  to  give  in  to  industry  clout  rather  than  fight  openly  to 
save  the  money  for  the  taxpayer  or  whatever. 

DR.  UHLANER:  In  a very  simple  answer,  yes.  You  have  to  think  beyond 
the  individual  case.  If  you  remove  yourself  from  the  system  you  can 
only  do  that  once.  You  might  be  able  to  take  a problem  higher  in 
DoD  and  that  might  or  might  not  accomplish  something,  but  going 
outside  the  system  completely  is  something  even  people  who  were 
sympathetic  with  your  views  regarding  a particular  case  might  be 
upset  about.  Someone  published  a book  in  the  last  year  on  what 
happened  to  people  who  resigned  from  the  government  over  matters  of 
principle.  This  was  an  historical  study  that  went  back  over  a con- 
siderable number  of  years  looking  at  various  people,  at  the  assistant 
secretary  level  and  higher,  who  had  done  that.  Uniformly  they  found 
themselves  shut  out  entirely  from  everything.  Beyond  that,  1 think 
it's  somewhat  irrelevant  to  this  particular  case  because  the  facts 
were  that  in  dealing  with  the  intelligence  community  and  photo- 
interpretation, what  we're  concerned  about  was  classified.  You 
couldn't  even  admit  that  the  U.  S.  Government  owned  a Brownie  camera, 
so  that  this  would  have  been  particularly  inappropriate  to  try  to 
get  at  through  publicity. 

DR.  HAVELOCK:  Would  publicity  have  been  necessary  before  you  could 
turn  around  the  procedures  for  procuring  the  types  of  equipment  which 
R&D  recommended? 

DR.  UHLANER:  To  the  internal  community  I think  it  was  very  widely 
disseminated.  I don't  think  that  was  really  a problem. 

DR.  HAVELOCK:  But  it  was  protected  from  communication  to  key 
congressmen? 

DR.  UHLANER:  I would  have  to  say  yes.  That's  a general  issue  we 
might  want  to  discuss  at  some  point.  It  may  be  that  the  people 

within  the  DoD  community  have  a different  view  of  their  customers 
than  the  academics  here,  and  it's  not  an  issue  we  have  really  talked 
about  except  in  the  last  few  minutes. 

DR.  GLASER:  Let's  move  on  to  the  next  group  and  their  case.  What 
cases  did  your  group  take? 
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DR.  HAVELOCK:  We  took  two  cases.  We  started  with  the  REALTRAIN  case. 
One  of  the  problems  with  this  case  was--in  a sense--that  there  wasn't 
a problem.  That  is,  there  appeared  to  be  an  innovation  that  was 
successfully  adopted,  with  some  relatively  minor  qualifications,  and 
the  whole  matter  is  proceeding  on  course.  If  there  was  a significant 
problem  it  was  probably  in  the  area  of  premature  adoption;  that  is,  it 
was  implemented  without  waiting  to  see  if  all  the  necessary  conditions 
for  sustaining  it  had  been  achieved.  Probably  there  were  two  reasons 
for  this.  One  reason  was  that  there  was  a product  champion  who  be- 
caine  involved  early,  got  excited  early  and  pushed  it.  But  I think  in 
addition  to  that  there  was  a great  deal  of  push  from  the  innovation 
itself.  It  was  highly  visible;  the  fact  that  the  early  tests  involved 
a lot  of  personnel  in  the  field  made  it  impossible  to  hide  this 
thing  until  it  was  ready  for  a full  distribution  and  application. 

It  was  a very  attractive  thing  to  a lot  of  people,  particularly  at  the 
top,  not  necessarily  at  lower  levels,  and  that's  where  they  ran  into 
some  problems  of  implementation. 

This  case  did  bring  up  some  interesting  problems,  some  of  which  we 
think  may  be  generic  to  military  innovations.  One  is  that  there  was 
a resistance  by  old  believers,  field  commanders  who  just  felt  com- 
fortable doing  it  their  own  way.  Many  of  those  v/ho  had  done  training 
in  the  field  believed  in  the  old  way  to  do  it,  so  when  they  got  the 
new  techniques  what  happened  in  part  was  that  they  did  half-assed 
implementation,  and  that  created  a self-fulfilling  prophecy  situation. 
But  in  effect,  my  inference  was  that  this  problem  is  being  dealt  with; 
that  is,  the  system  is  convincingly  superior  for  those  who  do  try 
it  out,  and  that  it  gradually  is  becoming  a pervasive  method  in  the 
Army.  We  didn't  talk  about  transfer  to  the  Marines  or  any  other 
servi ce. 

Another  problem  which  is  connected  to  this  is  the  problem  of  stringent 
adoption.  This  was  a complex  innovation  which  in  effect  had  to  be 
adopted  in  its  entirety.  It  lacked  what  Everett  Rogers  calls 
"divisibility"  of  an  innovation.  It  wasn't  something  where  you 
could  pick  up  this  piece  and  substitute  it  for  that  piece  and  adapt 
in  your  own  way.  It's  a very  human  thing  to  do;  that  is,  most  user^ 
like  to  pick  and  choose--they  want  to  be  in  command  of  the  adoption 
process  and  it's  really  an  adaptation  process.  But  there  are  some 
kinds  of  high  technology  innovations  where  if  you're  too  creative  in 
your  adoption  you  can  destroy  the  whole  thing.  There  are  certain  key 
points  in  any  innovative  system  where  when  you  adapt  it  instead  of 
adopt  it,  you  screw  it  all  up.  With  the  premature  adoption  there 
was  a logistics  lag;  l.e.,  there  were  all  kinds  of  things  that  they 
didn't  have  ready,  so  that  there  was  a risk  in  massive  and  early 
adoption  even  with  a good  thing.  In  fact,  if  the  right  equipment 
didn't  come  in  and  people  were  horsing  around  with  the  wrong 
procedure,  it  may  not  have  looked  like  the  good  thing  it  really 
was . 

Another  kind  of  issue  which  we  found  applied  to  some  extent  in  the 
other  case  we  looked  at  is  the  tendency  of  Americans  to  opt  for 
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the  highest  technology  we  can  find  at  the  time  to  do  something  which 
may  be  possible  to  accomplish  with  simpler  technology.  When  this 
innovation  got  to  the  application  stage,  particularly  when  it  got  to 
the  point  of  building  a system  for  tank  training  rather  than  just 
infantry  training,  which  was  the  original  model,  the  guys  then  began 
thinking  of  lasers  and  other  kinds  of  esoteric  stuff.  It  got  to  be 
so  technically  elegant  that  the  probability  of  the  system  breaking 
down  or  screwing  up  in  some  way  was  increased  logarithmically.  I 
guess  we  are  still  working  through  that  kind  of  issue.  The  need  to 
make  something  perfect  or  very  elegant  can  be  a significant  utiliza- 
tion problem  One  possibility  was  when  the  generals  insisted  that 
there  be  immediate  application,  the  researchers  might  have  said, 

"O.K.,  if  we  are  going  to  go  ahead  with  that  let's  increase  our 
budget  for  evaluating  the  thing  so  we  can  keep  track  of  this  fully 
as  it  goes  along."  The  general  said  you  can  keep  doing  what  you're 
doing  and  that's  fine  but  sorry,  we  can't  give  you  more  money.  The 
results  of  that  evaluation  aren't  in  as  yet  in  terms  of  whether  this 
is  going  to  create  a cybernetic  type  of  feedback  loop  that  will  lead 
to  a continuing  self-improvement  of  this  new  system. 

In  our  discussion  the  question  came  up  about  who  should  control  the 
adoption  decision.  In  effect,  it  was  controlled  in  a kind  of  dialogue 
process,  with  the  generals  obviously  making  the  final  decision. 

The  suggestion  was  made,  that  is  related  to  our  question  of  cost- 
effectiveness  models;  that  we  don't  really  have  a utility  model  for 
deci s i on-maki ng . 

(The  tape  recording  of  the  remainder  of  the  case  problem  session 
contained  many  gaps  and  unintelligible  segments.  We  have  done  our 
best  to  recreate  the  session,  and  apologize  for  any  distortions  that 
we  may  have  injected.) 

At  this  point.  Dr.  McClelland  reported  his  group's  discussion  of  the 
Navy  case  problem.  His  points  included  the  following: 

I 

1.  This  case  illustrates  the  naivete  of  laboratory  people.  For 
example,  they  apparently  made  no  real  effort  to  determine  the 
comparability  of  the  two  naval  training  schools  that  trained 
for  this  rate.  It  would  be  important  to  know  the  extent  to 
which  they  were  equivalent  in  terms  of  such  variables  as 
talent,  technology,  leadership  climate,  morale,  and  so  forth. 

2.  Little  attention  was  apparently  paid  to  the  cost  of  implementing 
the  heroic  procedures  for  remediating  the  attrition  rate.  The 
researchers  got  carried  away  to  the  point  that  cost  and  effort 
(e.g.,  working  nights)  became  no  object.  A cost-effectiveness 
analysis  should  routinely  be  part  of  the  technical  research 
report . 

3.  Adequate  provision  should  be  made  for  "selling"  promising 
research  results.  Resources  must  be  allocated  for  research 
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utilization,  and  such  resources  should  probably  include  research 
utilization  specialists. 

DR.  UHLANER;  You  made  a comment  that  researchers  aren't  the  best 
people  for  implementing  researcli  results.  I would  like  to  ask 
whether  you  know  of  any  system  where  it  works  more  effectively  to 
have  different  groups  for  research  and  utilization.  I think  you 
should  examine  that  statement  very  carefully,  because  in  all  the 
systems  I know  of  there  is  someone  in  the  research  community  who 
really  takes  the  lead  in  implementation.  I don't  know  of  any 
system  that  has  separate  research  implementors.  If  anyone  does, 
and  it  works  well,  I'd  like  to  hear  about  it. 

LCDR  HAIRS:  At  the  Bureau  of  Naval  Personnel  there  is  a scientific 
advisor  who  has  a staff  consisting  of  himself  and  a secretary.  That's 
an  absolutely  impossible  situation.  We  decided  he  should  have  a 
technical  staff  of  at  least  12  people.  His  task  would  be  to  review 
the  research  that  comes  in  and  assume  some  sort  of  quality  controi 
function.  (l  talked  to  Glenn  Bryan  about  the  case  load  that  he  has 
at  ONR  ?^d  I was  astounded  that  it's  so  high.  It's  impossible  to 
keep  on  top  of  something  like  30  to  35  technical  reports  a year  that 
he  might  be  responsible  for  going  through.  I myself  have  difficulty 
skimming  aoout  100  journal  articles  a year  and  I doubt  if  I get 
into  detail  on  more  than  two  or  three,  yet  it  has  to  be  done.)  At 
BUPERS  if  the  scientific  advisor  had  an  adequate  staff,  that  cut 
across  the  major  divisions  within  BUPERS,  then  we  would  know  that 
they're  the  people  to  go  to  in  order  to  get  something  implemented. 

I'm  sure  that  no  one  before  this  meeting  knew  that  if  they  had  a 
problem  that  had  something  to  do  with  saving  money  there  is  an 
economic  analysis  section  that  can  take  on  that  kind  of  task.  We 
need  to  have  such  resources  available  to  a majority  of  the  programs. 

I have  been  beating  on  the  R&D  community  to  use  that  kind  of  help, 
and  I intend  to  do  the  same  thing  with  the  operators.  That  is,  we 
need  to  identify  resources  that  try  to  service  this  interface  function 
and  can  take  a project  and  try  to  get  it  implemented. 

DR.  BERKHOUSE:  For  thirty  years  now  I have  been  doing  research  and 
I have  had  my  research  implemented  in  some  forms  where  it  has  been 
used  quite  sensibly.  I have  also  witnessed  some  research  that 
never  got  implemented  at  all. 

We  have  three  different  kinds  of  people  involved  in  research.  The 
first  two  are  very  clo' ■“  together.  One  is  the  man  who  does  techno- 
logical base  research.  His  assignments  are  such  that  he  isn't 
really  very  dedicated  to  all  the  fine  details  of  trying  to  get  his 
research  utilized,  but  he's  the  most  valuable  member  of  the  team. 

The  second  type  is  the  guy  who  is  good  at  developing  and  conducting 
research  that  results  in  a usable  product.  He's  very  pragmatic.  These 
two  groups  overiap,  and  one  individual  can  be  involved  in  both  groups 
simultaneously,  or  move  from  one  to  the  other.  Then  we  have  a third 
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situation,  which  requires  different  kinds  of  people.  It  calls  for 
primarily  officer  personnel  who  should  have  a good  technical 
background,  at  least  a masters  degree,  should  be  interested  in 
research,  understand  research,  but  also  have  their  finger  on  the  pulse 
of  the  operating  organization  that  is  going  to  use  the  research.  They 
would  be  teamed  up  with  scientists,  but  the  scientists  would  not  be 
permanent  members  of  this  group.  If  you  rigidly  establish  such  a 
group,  you're  sunk.  It  has  to  keep  changing  in  accordance  with  the 
needs  at  the  time. 

I believe  there  is  a need  for  this  third  group.  I believe  the  members 
of  this  group  should  be  involved  with  research  before  it  is  begun. 

They  must  somehow  get  busy  and  perform  an  evaluation  of  the  initial 
research  requirement  and  render  a judgment  at  that  point  as  to 
whether  it  is  likely  that  it  can  be  utilized.  They  must  also  develop 
a blueprint  for  the  plans  to  execute  the  implementation  and  utiliza- 
tion of  the  research.  They  must  also  constantly  review  this  and  be 
aware  of  utilization  potential  as  the  research  goes  on,  so  that  two  or 
three  years  later  when  the  scientist  is  finished  with  his  research 
product  this  group  can  get  busy  and  program  its  utilization. 

This  group  must  somehow  have  representation  from  general  staff  agency 
level  or  major  command  level.  It  must  also  have  access  to  the  lower 
commands  that  constitute  the  users.  This  group  must  develop  a blue- 
print for  what  I'm  going  to  call  "salesmanship."  1 don't  like  the 
word  salesmanship  but  that  is  essentially  what  it  is.  I have  about 
finished  the  sermon,  but  we  do  need  an  additional  approach  to  our 
program  from  what  we  have  had  heretofore.  This  cannot  be  rigid; 
it  must  be  a flexible  approach. 

DR.  GLASER:  Do  we  have  any  more  comments  on  this  case? 

COMMENT:  We  also  discussed  this  case  and  1 think  it  has  been  covered 
so  well  that  I don't  have  to  say  much  more  about  it.  One  point  that 
particularly  struck  me  has  already  been  mentioned.  The  impression 
you  get  is  that  the  reduction  in  attrition  wasn't  so  spectacular  in 
light  of  all  the  resources  that  were  put  into  it.  For  example, 
there  were  people  working  nights  and  I wonder  what  cost  was  involved 
i n that . 

DR.  HAVELOCK:  We  had  a similar  observation.  What  happened  to  the 
researchers  in  this  case  was  not  unlike  what  happened  to  the 
engineers  in  the  other  case  when  they  were  trying  to  adapt  the  system 
for  use  with  tanks.  That  is,  they  got  carried  away  and  wanted  to 
implement  the  thing  in  the  most  elegant  form  possible,  without 
considering  the  cost  involved. 

DR.  CRAWFORD:  Our  group  also  took  this  case.  It  is  a nutritious 
object  of  attention  because  it  has  so  many  ramifications,  such  as  the 
fact  that  both  selection  and  training  treatments  were  used,  and  you 
have  two  different  universes  there.  For  example,  it  is  probably 
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true  that  with  a very  careful  selection  technique  you  could  find 
people  who  would  meet  a proficiency  requirement  with  very  limited 
training.  But  then  you  get  into  the  question  of  how  aptitudes  are 
distributed,  and  whether  you  can  f i nd  the  aptitudes  that  are  re- 
quired for  that  particular  thing. 

Among  our  group  there  was  some  skepticism  as  to  whether  the  attrition 
could  be  reduced  by  that  much.  That  leads  to  the  question  first,  of 
how  proficiency  was  measured,  and  whether  there  was  agreement  between 
operator  and  researcher  that  the  terminal  performance  in  this  course 
was  representative  of  the  actual  requirements  of  the  job.  That 
was  not  stated,  but  it  would  be  very  important  in  the  selling  of 
any  training  system.  Finally,  as  has  already  been  referred  to,  the 
cost  in  terms  of  additional  training  equipment  and  particularly  the 
additional  hours  the  instructor  put  in  needs  to  be  laid  out.  If 
the  commander  expected  someone  to  do  something  with  that  technical 
report  he's  got  another  think  coming;  he's  got  to  do  more  than  that, 
but  you  nevertheless  have  to  get  the  basic  information  somewhere,  and 
hopefully  it  is  in  the  technical  report. 

QUESTION:  Does  anyone  have  knowledge  of  whether  this  is  an  actual 
case  or  was  created? 

DR.  GLASER:  I have  knowledge  of  it.  Parts  of  the  case  describe  an 
actual  situation.  There  was  in  fact  a great  reduction  of  attrition 
in  the  situation  described,  but  some  of  the  figures  cited  are 
fictitious.  The  case  goes  back  to  a World  War  11  situation  involving 
a radio  training  school. 

(Because  the  discussion  of  the  Navy  case  problem  was  incompletely 
recorded.  Dr.  Glaser  prepared  the  following  summary  subsequent  to  the 
sympos i urn. ) 

Additional  information  needed  was: 

1.  In  a situation  where  the  Navy  had  two  schools  in  the  U.  S. 
giving  the  same  type  of  training,  and  attrition  at  one  was 
55-60%  while  only  38%  at  the  other,  why  didn't  the  research 
team  study  the  situation  at  both  schools  to  see  if  they  could 
account  for  this  difference? 

2.  Since  attrition  at  the  school  that  requested  help  rose  from 

30%  to  55%  following  abolition  of  the  draft,  did  the  researchers 
"dig  into"  possible  reasons  contributing  to  this  change?  For 
example,  did  the  trainees  entering  the  school  differ  significantly 
in  measurable  ways  from  those  who  had  entered  previously?  Did 
the  instructional  technology  change?  Was  there  major  turnover 
of  personnel  in  the  management  of  the  school;  e.g,.  Commanding 
Officer,  Executive  Officer,  instructional  staff?  Were  morale 
factors  clearly  different,  and,  if  so,  in  what  ways  and  for 
what  reasons? 
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Factors  that  might  be  relevant  to  the  question  of  why  there  was  no 
evidence  of  utilization  elsewhere  of  the  procedures  demonstrated  to 
be  successful  in  this  case  probably  were; 

1.  The  concern  and  influence  of  the  sponsor  of  this  piece  of 
research  (the  CO)  appeared  limited  to  his  own  situation.  If 
the  researchers  had  general i zabi 1 i ty  and  wider  utilization  of 
their  prototype  procedures  in  mind  they  should  have  suggested 
that  the  request  for  help  be  channeled  through  and  supported 

by  the  Training  Command  that  had  jurisdiction  over  all  training 
schools.  And  they  should  have  explored  the  possibility  of 
studying  both  schools  offering  this  type  of  training,  not  just 
the  one. 

2.  Since  the  outcome  of  this  R£D  effort  was  a reduction  of 
attrition  from  55%  to  only  22%  and  then  further  to  11%  after 
instituting  highly  individualized  instruction  and  personal 
counseling  of  trainees,  that  was  a dramatic  improvement  not 
only  in  relation  to  the  school  which  requested  the  study,  but 
to  the  other  school  which  still  had  a 38%  failure  rate.  Did 
the  researchers  then--albeit  belatedly-try  to  involve  the 
other  school  in  a similar  study  and  see  if  the  same  procedures 
might  result  in  reducing  their  attrition  rate  significantly? 

Some  things  that  still  might  be  done  in  an  effort  to  promote  at 
least  cross-validation  if  not  widespread  utilization  of  the  R&O 
which  has  seemed  so  successful  in  one  case  might  be: 

1.  To  prepare  a very  readable,  interesting  executive  summary  report 
of  the  problem,  procedures,  findings,  interpretation  of  the 
findings,  and  policy  implications. 

2.  To  arrange  a briefing  for  the  Training  Command  about  these 
findings,  and  submit  a request  for  Training  Command  sponsorship 
to  replicate  the  procedures  in  some  other  appropriate  training 
si tuations. 

Some  general  implications  from  this  case  problem  would  seem  to  include: 

1.  Don't  jump  in  to  respond  to  a specific  request  for  R&D  service 
without  considering  who  the  potential  users  might  be  and  how 
you  might  be  able  to  obtain  sponsorship  for  the  study  which 
could  influence  those  potential  user  categories  if  the  results 
warranted  their  attention  for  possible  adoption. 

2.  Include  a cost-benefit  analysis  as  part  of  the  plan  for  a 
study  of  this  kind.  The  most  dramatic  results  seemed  to  come 
from  highly  individualized  attention  to  and  special  counseling 
with  each  student.  That  is  a costly  procedure,  and  not  easy  to 
bring  about.  What  in  fact  were  the  economic  and  other  costs 
involved  in  this  case? 
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3.  Don't  depend  on  just  publication  of  a report  to  influence  other 

potential  users.  Think  through  and  carry  out  other  more  personal- 
ized ways  of  attracting  the  attention  of  potential  users  to 
the  study  and  its  findings,  such  as  through  inviting  their 
consultation  on  interpretation  of  the  findings,  participation 
in  briefings,  making  visits  to  the  demonstration  site,  involving 
the  CO  of  the  school  to  work  with  the  researchers  in  inviting 
attention  through  h i s contacts  and  chain  of  command,  etc. 

DR.  BRYAN:  1 get  worried  about  the  research  community  standing  aloof 
from  the  bureaucracy  and  being  irritated  by  having  to  deal  with  it. 

And  Ron  Havelock  mentioned  that  on  some  occasions  the  bureaucracy 
fades  away.  My  plea  would  be--and  1 think  I'm  in  bed  with  Lee  Mairs 
on  this,  and  that's  rare--the  bureaucracy  often  does  not  fade  away. 

1 think  things  do  get  done  by  the  bureaucracy,  and  it  is  not  reasonable 
for  the  R&D  people  to  think  they  have  some  special  way  of  getting 
things  done.  They  have  to  learn  what  the  bureaucracy  is  and  have  to 
be  patient  with  it,  have  to  suffer  it,  have  to  work  through  it,  and 
if  you  do,  it  ordinarily  works  well  even  though  it  takes  a lot  of 
time  and  frustration.  One  of  the  problems  is  that  when  you  go  to 
the  door  of  the  bureaucracy  and  knock  on  the  door  you  find  either 
nobody  there  or  you  find  some  guy  who  is  a staff  assistant.  He 
thanks  you  very  much  for  coming  and  then  you  leave  it  in  his  lap, 
but  he  knows  less  of  what  to  do  about  it  than  you  do  because  the 
next  guy  who  knocks  on  the  door  has  a whole  new  set  of  things  he's 
going  to  try  to  dump  on  him.  1 think  part  of  the  problem  is  that 
the  bureaucracy  and  the  R&D  people  are  not  sufficiently  acquainted 
with  each  other.  And  the  R6D  people  do  not  feel  that  the  bureaucracy 
is  there  for  them  to  use.  My  feeling  is,  that  is  what  the  bureaucracy 
is  there  for,  and  when  we  have  "used"  it  properly  it  tends  to  serve 
us  rathe r well. 

DR.  GLASER:  Dr.  Rostker  is  interested  enough  in  the  hoped  for  policy 
recommendations  from  this  group  that  he  is  coming  back  this  afternoon 
and  tomorrow.  Here's  a decision  maker  who  is  in  control  of  funds,  and 
is  hoping  to  get  some  recommendations  that  are  constructive  as  to 
what  might  be  done  better  in  terms  of  payoff  from  R&D  investment 
so  that  he  might  be  in  a better  position  to  program  funds. 

DR.  SANDS:  Some  people  missed  his  banquet  address,  so  we'll  play 
the  audio  tape  tomorrow.  1 think  it's  important  to  take  this  into 
your  groups. 

DR.  DRUCKER:  We  talked  about  the  Navy  case  and  the  stereoscopic 
problem.  There  are  things  that  were  most  prominent  in  our  discussions: 
We  felt  the  need  for  user  commitment  in  many  of  the  cases  that  we 
read,  and  along  with  that  a need  for  researcher  commitment  with 
respect  to  utilization.  To  put  it  a different  way,  we  called  it 
a user-researcher  agreement  which  ideally  is  reached  at  the  start 
of  the  research.  We  found  such  a user-researcher  agreement  singularly 
lacking  in  at  least  three  of  the  case  studies.  We  also  talked  about 
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the  need  for  front-end  analysis,  including  such  things  as  cost- 
benefit  estimates,  risk  analysis,  alternate  solution  analysis,  task 
analysis,  and  requirements  analysis. 

We  hit  on  the  need  for  an  awareness  of  systems  with  a large  S and 
the  implications  of  changing  one  part  of  the  large  system  without 
paying  sufficient  attention  to  what  it  might  do  to  another  part  of 
the  system.  We  need  to  sharpen  our  awareness  and  focus  on  the  total 
system  implication  of  what  we  do  or  are  asked  to  do,  as  nicely 
illustrated  in  the  Navy  case. 

We  also  need  to  beware  of  our  tendency  to  get  too  sophisticated  in 
our  innovations  when  we  do  not  have  to,  as  illustrated  by  the  Armor 
School  and  REALTRAIN  cases.  There  is  a need  to  institutionalize  our 
change  procedures. 

DR.  CRAWFORD:  The  requirement  to  follow  the  Armed  Forces  Vocational 
Aptitude  Data  is  still  there  and  we  need  to  get  as  much  validation 
data  as  possible. 

DR.  SMITH;  One  last  comment  on  the  REALTRAIN  case:  For  some  years 
now  the  Navy's  6.2  programs  quoted  the  work  on  laser  and  holographic 
techniques  used  in  training  devices  at  the  Navy  Training  Equipment 
Center.  Thus,  the  effort  by  the  Army  in  the  use  of  lasers  that  was 
labeled  as  excessive  was  in  fact  an  instance  of  the  utilization  of 
Navy-developed  R&D  technology  in  an  Army  setting. 
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SELECTED  CONC|PT  _PAPEKS_ 


SUMMARY 


In  this  session  two  aonaept  papers  were  presented  that  had  been 
prepared  for  the  symposium  to  highlight  issues  in  research  utili- 
zation. 

The  first  presentation  was  by  Merlin  Malehom,  Office  of  the 
Director,  Naval  Education  and  Training,  and  was  based  on  his  paper 
entitled  "Bridge  Building. " The  focus  of  the  presentation  was  on 
efforts  by  the  Education  and  Training  Command  of  the  Navy  to 
facilitate  utilization.  He  noted  that: 

. Research  utilization  is  too  complex  a process  to  be  reduce- 
able  to  a paper  model  or  regulation. 

. Operators  do  not  do  a good  job  of  stating  requirements, 
tending  to  be  too  global,  to  ask  for  things  that  already 
exist,  and  to  specify  solutions  rather  than  problems. 

. The  process  of  formulating  requirements  is  complex  and  re- 
quires ongoing  dialogue  between  users  and  producers. 

. There  is  need  for  careful  planning  of  end  products  and  their 
implementation,  beginning  early  in  the  research  program  and 
frequently  iterated. 

. A technical  report  is  not  an  adequate  end  product  in  most 
instances. 

. Users  fail  to  take  advantage  of  opportunities  for  prototype 
development. 

The  second  presentation,  by  Melvin  Schwartz,  Navy  Personnel  R&D 
Center,  was  based  on  the  paper,  "The  Utilization  of  People- 
Related  Research  and  Development:  A Case  History  of  the  Shipboard 
Facilities  Maintenance  Demonstration  Study,"  by  Corder  and  Schwartz. 
He  discussed  a research  project  designed  to  improve  shipboard  facil- 
ities maintenance  while  reducing  manning  levels,  and  commented  on 
factors  which  appear  to  affect  utilization  of  results.  He  stressed 
the  importance  of  developing  ad  hoc  teams  of  sponsors,  users,  study 
participants  and  researchers  for  such  purposes  as  establishing  the 
study  objective,  preparing  specific  operational  requirem,ents , de- 
veloping the  study  approach,  determining  logistics  support  require- 
ments, designing  evaluation  procedures,  communicating  findings,  and 
planning  system  implementation. 
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) CONCEPT  PAPERS 


DR.  SANDS:  I would  like  to  introduce  Mr.  Merle  Malehorn,  the  Deputy 
Director  of  Education  and  Training  Programs  Division,  Office  of  the 
Director  of  Naval  Education  and  Training. 

"BRIDGE  BUILDING" 


Mr.  Merlin  K.  Malehorn 

Deputy  Director,  Education  & Training  Programs  Division 
Office  of  the  Director  of  Naval  Education  and  Training 


In  view  of  some  of  the  comments  that  have  been  made  here  let  me  say 
that  I am  bureaucracy  and  so  are  you.  In  fact,  I am  a mission 
sponsor  for  education  and  training  and  that  also  makes  me  a mission 
sponsor  for  some  parts  of  Navy  R&D  as  related  to  the  training  enter- 
prise. I don't  know  what  that  classifies  me,  maybe  a middleman,  or 
an  R&D  manager  or  a user  or  whatever.  I thought  I was  a user  for 
a while,  but  now  I'm  not  sure  that  is  true.  And  I am  not  sure  that 
I really  know  who  the  user  is,  the  more  I think  about  it.  Are  we 
talking  about  the  school,  are  we  talking  about  the  echelon  that 
determines  what  the  school  shall  do,  or  are  we  talking  about  those 
of  us  who  set  requirements  for  what  the  school  shall  provide?  I 
have  a problem  defining  the  usei — and  I guess  I even  have  problems 
knowing  what  we  mean  by  researcher.  The  real  title  of  my  paper  is 
something  else. 

I have  invented  a new  unit  or  a new  name  or  a new  activity:  robelings. 
Robelings  are  the  activities  that  one  engages  in  to  try  to  get  the 
user  to  do  something  with  the  research  or  product  you  have  created. 

I did  that  because  in  many  cases  it  seems  that  we  are  trying  to 
sell  the  Brooklyn  Bridge.  What  I want  to  do  this  afternoon  is 
tell  you  some  of  the  things  we  are  trying  to  do  in  the  Education 
and  Training  Command  of  the  Navy  to  improve  our  own  posture,  in 
terms  of  defining  our  requirements,  in  terms  of  improving  the 
dialogue  with  the  R&D  community  itself,  and  in  terms  of  improving 
the  dialogue  between  the  two  communities.  In  that  sense  we  also 
are  trying  to  move  research  results  into  utilization  in  some  way. 

I have  a sort  of  self-conceived  model,  probably  pretty  naive,  and 
I have  not  drawn  it  out  as  a model  because  I do  not  like  to  use  that 
term  for  things  that  need  to  be  done.  We  in  the  Education  and 
Training  community  started  a year  ago  pursuing  a variety  of 
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activities  that  fit  in  my  undisclosed  model  and  have  a list  of  other 
things  that  can  be  done,  but  we  are  interested  in  engaging  in  them 
as  activities  rather  than  as  part  of  a model. 

I have  a tendency  not  to  like  models.  Both  the  Army  and  the  Air 
Force  have  commented  that  they  have  regulations  that  deal  vnth  this 

information  transfer  process  or  whatever  you  want  to  call  it.  We 

have  one  in  the  Navy  too,  it's  the  CNET  Instruction  3910. lA, 
and  it  doesn't  work  so  we  don't  talk  about  it.  The  reason  it  doesn't 
work  is  that  it  is  too  stylistic  and  formalistic  and  constraining.  My 
experience  in  trying  to  promote  this  business  in  the  last  year  or  two 

indicates  to  me  that  it  must  be  too  complex  a process  to  really  be 

susceptible  to  relatively  simple  description  on  a piece  of  paper.  If 
what  appears  on  the  paper  is  perceived  as  too  complex,  people  don't 
use  it.  So,  we  are  engaging  in  a series  of  activities. 

What  I will  discuss  then  or  describe  for  you  briefly  is  not  everything 
we  do  but  some  of  the  things  we  do.  I wi 1 1 wander  at  random  through 
a variety  of  subjects  or  categor i es "“We  will  talk  about  various 
pieces,  and  some  other  things  also.  I will  not  follow  the  paper  I 
have  submitted  in  advance  of  this  meeting,  so  you  had  better  read  that 
too.  In  the  Navy  when  we  have  tried  to  get  together  those  of  us 
who  are  users,  those  of  us  who  are  researchers,  and  those  of  us 
who  are  some  kind  of  go-between,  the  thrust  of  the  discussion  has 
been,  "Gee,  you  users  ought  to  state  your  requirements."  We  keep 
trying  to  do  that.  I wish  it  were  that  simple.  So  one  of  the 
activities  we  are  engaging  in  is  to  try  to  improve  upon  the  manner 
in  which  we  state  our  requirements.  However,  my  general  evaluation 
has  been  that  we  who  are  users  tend  to  want  our  programs  immediately 
and  we  are  not  willing  to  wait  for  research  per  se  to  be  done  in  the 
sense  the  researcher  thinks  of  the  term  research.  We  want  a reading 
skill  program  and  we  want  it  tomorrow.  Kids  are  flunking  out  of  boot 
camp  because  they  can't  read,  at  least  we  think  that's  the  problem. 

We  also  tend  sometimes  to  v;ant  "global  fixes,"  like  our  requirement  to 
reduce  the  attrition  rate.  That  is  hardly  a definitive  requirement. 

I think  we  all  know  the  attrition  rate  is  too  high,  though  we're 
not  too  sure  of  what  it  should  be.  We  think  it  can  be  too  low  as 
well,  unless  we  make  all  our  entry  level  selection  criteria  adequate. 
Thus,  we  think  there  probably  is  some  necessary  lower  level  of  attrition. 

In  any  event,  we  tend  to  be  very  global  in  our  statements,  or  else 
we  tend  to  be  very  microscopic,  like,  "How  about  putting  programmed 
instruction  into  OCS,"  vyhich  from  the  operators'  standpoint  is  an 
important  problem,  but  from  the  research  standpoint  I suspect  is 
trivial,  although  I don't  really  know. 

Another  thing  we  tend  to  do  is  to  ask  for  things  that  are  really 
studies  rather  than  research,  and  here  my  problem  tends  to  be  diffei'ent 
as  a manager.  If  in  all  good  conscience  we  ask  the  training  school 
level  or  command  level  and  others  like  that  to  identify  their  research 
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requirements,  we  find  that  they  are  not  really  accustomed  to  that 
type  of  thinking.  I think  it  would  be  unfair  to  ask  everybody  at  all 
levels  to  be  schooled  in  the  rather  nice  definitions  of  what  con- 
stitutes research,  development,  test  and  evaluation.  So  they  come 
in  with  sets  of  problems,  and  many  of  those  really  at  best 
necessitate  a "study"  of  some  kind,  and  maybe  all  they  really  require 
is  consultation.  What  tends  to  happen  with  us  is  that  the  problems 
that  really  require  research  are  sort  of  thrown  in  the  corner.  We 
don't  really  attend  to  them;  management  is  not  sensitive  to  them  to 
the  same  degree  that  they  are  sensitive  to  consultation  needs. 
Ultimately,  therefore,  we  have  negative  feedback,  and  before  you 
know  it,  school  level  people  are  not  talking  to  us  because  we  have 
not  really  attacked  their  problems. 

Another  thing  the  operators  tend  to  do  is  ask  you  to  develop  an 
application  of  something  that  already  exists.  For  example,  we  got 
from  Second  Fleet  a requirement  for  an  EW  environment  simulator.  It 
turns  out  that  the  thing  is  on  the  shelf,  you  can  buy  it;  no 
development  is  required.  Nonetheless,  it  was  an  operational  re- 
quirement as  far  as  he  was  concerned.  What  we  had  to  do  was  tell 
him  he  was  talking  to  the  wrong  community.  It  wasn't  an  R£D 
problem,  he  needed  to  talk  to  the  procurement  people.  Fortunately, 
we  got  the  thing  into  the  hands  of  the  procurement  people  and  something 
was  done  about  it.  The  point  is  that  we  who  are  on  the  operational 
side  tend  to  talk  about  applications  that  already  exist  rather  than 
developmental  kinds  of  work.  We  also  tend  to  tell  you  the  solution 
rather  than  the  problem. 

We've  had  an  inquiry  from  one  of  our  commands  that  runs  correspondence 
courses;  they  want  us  to  set  up  an  evaluation  program  and  motivation 
program  that  will  make  people  who  request  correspondence  courses 
complete  them.  My  question  back  to  that  command  is,  "Why  do  you 
assume  that  correspondence  courses  per  se  are  a good  solution  to 
the  problem  to  start  with?"  I have  taken  a lot  of  them  and  I find 
them  to  be  horrible  solutions  to  the  learning  business. 

My  point  here  is  that  it  is  relatively  simple  to  say  that  the 
operator  ought  to  state  his  requirements.  It  does  not  happen  that 
easily;  it's  a never-ending  process  of  dialogue,  even  within  the 
operator-user  community. 

One  of  the  things  in  the  Navy  that  mission  sponsors  are  supposed  to 
have  are  long-range  plans,  and  given  those  long-range  plans,  the 
RSD  community  can  bang  their  existing  RED  programs  against  those  plans 
and  try  to  see  if  there  is  some  kind  of  fit.  We  do  not  have  a long- 
range  plan.  We  have  been  casting  about  within  the  Education  and 
Training  community  for  three  years  now.  trying  to  get  one  created, 
and  it  is  not  easy.  One  of  the  problems  is  that  there  is  a tendency 
to  want  to  write  a description  of  the  o'^gan  i zat  i on  as  it  now  exists 
rather  than  what  it  ought  to  be  perhaps  20  years  from  now.  Again, 
it's  a learning  process  and  it  takes  a lot  of  patience. 
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The  CNET  command  a year  or  so  ago  went  out  and  obtained  a long  list 
of  operational  requirements,  whl^ii  | have  changed  to  operational 
objectives  because  to  us  operational  requirements  means  a formal 
paper.  They  massaged  them  in  some  way  or  another  and  came  up  with 
three  pages  of  this  kind  of  thing.  Part  of  my  job  was  to  integrate 
and  synthesize  the  requirements.  The  point  is  that  one  of  the  ways 
of  getting  at  needs  or  capability  requirements  is  to  try  in  some  way 
to  exercise  professional  judgment  and  integrate  them.  We  are 
recycling  this  process  and  I must  talk  about  some  problems  this 
causes.  One  of  the  problems  is  that  it  was  not  done  by  systematic 
analysis.  It  was  an  opinion  survey  and  it  is  difficult  to  get 
from  that  to  a systematic  analysis  of  where  you  really  ought  to  be 
because  you're  getting  opinions  from  people  who  really  don't  know 
what  you  might  be  capable  of  doing.  Another  problem  is  that  a survey 
does  not  tell  you  of  the  net  changes  required.  Generally,  all  they 
say  is  that  we  want  to  be  able  to  do  so  and  so.  But  you  don't  know 
where  you  are  now  so  you  don't  have  any  kind  of  baseline,  and  you 
don't  know  when  you've  gotten  there  because  the  statements  are 
about  motherhood.  Without  a systematic  analysis  and  integration,  you 
can  be  trapped  into  doing  things  forever  and  ever  and  never  get  there. 

The  next  problem  is  a tendency  to  cast  about  for  something  new  and 
different,  something  that  has  not  been  tried  for  five  years.  I wen. 
through  some  of  the  R£D  projects  that  I was  aware  of  in  the  Navy  and, 
just  using  what  rules  I could  create,  I said  here  are  some  things  we 
may  be  able  to  accomplish  by  a given  time.  There  are  about  three 
pages  of  these.  Several  months  ago  we  got  together  some  of  the  user 
people  and  some  of  the  research  people  and  we  looked  at  these,  asking 
ourselves  whether  the  dates  are  any  good,  whether  the  terms  have 
any  meaning,  whether  we  need  to  add  to  it.  I must  say  that  was 
unproductive.  All  we  did  was  get  off  into  discussions  to  the  effect 
that  we  need  long-range  objectives  that  will  stand  forever,  such 
as  improving  upon  surface  warfare  training.  I agree  that  we  do  need 
to  improve  surface  warfare  training.  The  only  thing  is,  that  does 
not  give  me  any  kind  of  handle  on  what  to  try  to  manage  to  get  done. 

One  of  the  things  we  did  was  to  take  one  of  these  ideas  and  go 

through  a planning  exercise  to  actually  see  what  would  be  involved 
in  getting  to  a desired  capability  by  a given  date--to  see  what 
would  be  involved  in  driving  the  system.  I have  tried  to  phase 

the  work  in  terms  of  operational  capability  available,  in  production 

and  commencing  implementation.  The  idea  would  be  to  lay  out  the 
activity  necessary  in  R6D  and  on  the  operational  side  in  order  to  get 
to  a given  capability  by  a given  date.  Well,  that  has  problems.  It 
raises  Questions  along  these  lines:  How  do  you  qet  to  the  objectives? 
Who  are  the  players?  How  do  you  go  from  that  list  into  a plan  of 
action?  What  are  the  milestones  for  implementation?  Are  the  dates 
any  good?  Are  there  real  requirements  in  this  particular  case  or 
do  they  simply  reflect  what  the  R&D  projects  are  and  therefore  it's 
a case  of  R£D  driving  the  requirements? 
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I don't  think  the  process  of  setting  requirements  is  simplified  by 
having  the  operators  do  it  rather  than  the  researchers.  I think  there 
is  a need  for  a continuing  dialogue  in  the  process  of  setting  re- 
quirements; it's  much  too  simplistic  to  assume  that  we  operators  can 
sit  down  and  magically  create  requirements. 


This  brings  up  some  other  problems.  When  we  begin  to  go  into  the 
requirements  business  we  find  it  necessary  to  engage  increasingly  in 
a whole  lot  of  things  that  can  be  termed  front-end  or  early-on 
analysis;  such  things  as  cost-benefit  analysis.  I have  some  problems 
with  that  concept,  because  it  seems  to  me  that  we  do  not  relate  it 
to  readiness.  We  relate  it  to  dollars,  but  it  may  be  that  I have 
to  pay  a lot  of  dollars  to  get  the  last  increment  of  personnel 
readiness  squeezed  out  of  the  training  system,  and  that  increment 
may  be  critical.  I grant  you  that  I do  not  know  how  to  measure 
units  of  proficiency  or  even  units  of  readiness,  but  I don't  think 
that  cost-benefit  analysis  should  be  allowed  to  override  the  need 
to  do  that.  That  is  not  to  say  we  should  not  do  cost-benefit 
analysis.  We  need  to  analyze  the  requirement;  we  need  to  look  for 
alternative  solutions.  Too  many  times  the  solution  is  picked  for 
us,  particularly  in  the  6.^  program  which  we  use  to  build  prototypes. 
The  solution  is  given  to  us  by  some  mission  sponsor.  You  can  say 
how  about  going  to  low-cost  trainers,  but  you  don't  want  to  do  that 
even  if  you  can  show  him  in  dollars  that  that's  the  way  to  go.  He's 
a big  boy.  So  you  probably  say,  go  up  in  the  bureaucracy.  That 
doesn't  do  you  much  good  because  he  is  the  end  of  the  line  for  all 
practical  purposes. 


Another  problem  is  that  there  is  a need  for  task  analysis  once  you 
figure  out  what  you  are  going  to  try  to  accomplish,  and  there  is 
need  for  such  things  as  utility  analysis  and  economic  analysis  and 
feasibility  analysis.  I don't  know  what  all  these  terms  mean  but  we 
keep  doing  them  and  people  keep  asking  us  for  them.  I'm  never  sure 
what  the  terminologies  are  in  terms  of  how  they  differ. 

Another  thing  we  have  gotten  into  is  a need  to  do  ADP  risk  analysis 
when  computers  and  computer  software  are  involved  in  our  projects, 
because  we  cannot  spend  R6D  dollars  to  buy  ADP  software  unless 
there  is  an  ADP  risk  development  analysis,  so  we  can  get  involved  in 
that . 


So  there  is  a real  problem  in  the  front-end  piece  of  the.  business, 
in  establishing  requirements. 

We  are  at  the  present  time  cycling  both  the  statement  of  operational 
objectives  and  the  calendar  date  objectives  back  through  the  system 
again  in  two  different  ways  with  an  overlapping  set  of  players  to 
try  to  combine  the  two.  We  are  not  going  to  get  them  completely 
combined  but  we  are  now  in  the  second  stages  of  the  cycle  to  get 
them  together.  The  first  stage  was  to  do  them  separately.  Now 
we  are  going  to  try  to  hold  them  together.  Gradually--!  figure 
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in  another  three  or  four  years--we  might  have  a fair  statement  of 
some  of  the  capabilities  that  we  in  the  Education  and  Training 
Command  of  the  Navy  want  to  try  to  get  to,  with  some  reasonably 
definitive  statements  and  some  fairly  good  dates  that  are  rational, 
not  pulled  out  of  the  air. 

Another  thing  we  are  doing  increasingly  is  impiementat ior  planning, 
and  I will  talk  about  this  from  several  different  angles.  It  has 
seemed  to  us  that  in  many  cases  (and  I think  of  that  Navy  case  problem 
example  that  we  were  looking  at  this  morning)  the  research  gets  done 
and  it  gets  captured  in  a technical  report--maybe  a year  or  two  later 
--and  that  is  often  the  end  of  it.  There  is  an  assumption  that 
somehow  or  another  it's  going  to  get  picked  up.  Well,  even  if  it 
does  get  picked  up  and  there  is  enthusiasm  within  the  training 
community,  you  are  fighting  a two  to  three  year  resources  cycle. 

We  need  to  plan  right  now  for  promising  research  that  will  be  available 
for  implementation  commencing  in  I98O.  The  resources  must  be  planned 
now  because  we  will  be  going  into  POM  80  this  fall,  and  if  they 
are  not  in  POM  80  they  aren't  going  to  be  in  budget  80 , they  aren't 
going  to  be  in  fiscal  80  without  traumatic  reprogramming  and  traumatic 
reprogramming  within  the  Navy  Education  and  Training  Program  in  the 
Training  Command  hardly  ever  exists.  It  can't  be  done;  there  is  not 
that  kind  of  flexibility. 

This  spring  we  started  to  pick  a couple  of  projects  that  look  like 
they  are  coming  out  and  could  be  implemented,  and  are  trying  to  lay 
out  a plan  of  action  and  milestones  to  implement  the  results  of 
those  projects  starting  in  Fiscal  Year  80.  I should  point  out  that 
we  are  not  talking  just  about  resources.  Those  aspects  of  imple- 
mentation that  involve  acquisition  of  resources  focus  on  FY  80 , but 
there  are  a lot  of  other  actions  that  also  are  necessary. 

For  example,  the  training  command  has  to  write  triple  R's  this  fall  if 
it  wants  the  resources  in  POM  80.  When  we  get  our  plan  of  action 
and  milestones  written  in  both  of  these,  there  will  be  a whole  long 
sequence  of  steps  that  have  to  be  taken  and  written  again  each  year. 

The  idea  here  is  to  get  the  research  people  together  with  the  ultimate 
user,  who  happens  in  this  case  to  be  the  staff  members  of  the  Chief 
of  Naval  Technical  Training.  They  really  govern  what  goes  on  in  the 
schools,  about  which  we  are  talking.  But  to  get  them  together  and 
lay  out  a fairly  specific  plan  for  implementation  in  terms  that  the 
training  people  understand  becomes  a learning  process  for  the  re- 
searcher, not  for  the  training  people.  The  training  people  understand 
the  business  of  acquiring  o&M  and  getting  more  instructor  billets  and 
changing  curricula  and  media  selection  and  getting  the  Chief  of 
Naval  Education  and  Training  Support  to  budget  for  simulators,  and 
so  forth.  So  it  becomes  a learning  process  for  the  researchers. 

Interestingly  enough,  we  have  a working  group  set  up  on  each  one  of 
those  projects  and  each  working  group  is  headed  by  a member  of  the 
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research  community.  So  they  will  be  forced  to  learn  whether  they 
like  it  or  not,  I suppose  you  could  say.  We  are  seeing  how  it 
evolves,  and  whether  it  works.  So  far  it's  rather  promising,  although 
we  expect  lots  of  problems  on  the  low  end  of  the  learning  curve  on 
both  sides.  But  if  it  works  we  will  begin  to  try  to  expand  this 
process  and  apply  it  in  other  areas  where  it  seems  to  be  an  approp- 
riate process.  We  think  we  are  probably  going  to  have  a meeting 
with  the  Chief  of  Naval  Education  and  Training  Support,  who  is  a 
different  command  and  who  handles  the  computer  and  instructional 
system  development  program  for  what  we  call  our  Class  A schools 
and  controls  a lot  of  other  support  things  such  as  media  development, 
media  acqu i s i t i on-- these  kinds  of  things.  We  think  we  will  probably 
take  the  research  community  and  go  meet  vn  th  his  staff  next  time 
and  identify  some  more  projects.  The  key  here  was  that  we  brought 
the  research  community  and  the  training  program  coordinators  to- 
gether. It  was  not  guys  up  at  my  level  and  it  wasn't  people  at  the 
Chief  Education  and  Training  level,  it  was  the  worker  beesi 

Well,  that  creates  lots  of  problems  for  us.  What  projects  are  we 
going  to  talk  about?  You  can't  talk  about  all  of  them.  We  briefed 
on  about  six  down  there  that  we  were  just  not  ready  to  talk  about, 
although  some  of  them  are  ready  for  implementation.  Who  are  the 
players  that  have  to  be  involved  in  each  case?  It  tends  to  be  a 
unique  set  in  each  instance,  which  gets  me  back  to  one  of  my 
earlier  points--that  it  is  very  difficult  for  me  to  conceive  of 
a documented  model  that's  going  to  address  everything,  such  as  what 
kind  of  qualifications  the  players  have  to  have.  Not  every  re- 
searcher is  capable  of  doing  this  kind  of  thing,  not  every  training 
program  coordinator  has  got  the  patience  to  do  it.  So  there  is  a 
personnel  selection  problem.  What  kind  of  people  are  you  going  to 
get  involved  in  this  kind  of  dialogue?  We  have  a problem  because 
it  creates  additional  work  load  for  everybody-for  the  TPC  as  well 
as  the  researchers.  It  either  slows  down  work  or  makes  people  work 
overtime  or  makes  something  else  drop  out,  and  you  sort  of  have  to 
assume  that  everything  else  has  got  priority  or  they  wouldn't  be 
doing  it  now . 

In  some  cases  there  is  a question  of  whether  we  know  enough  about 
the  project  or  trust  the  findings  enough  and  that  becomes  another 
problem  in  evaluating  whether  to  go  with  it.  That  brings  up  still 
another  question  that  we  have  not  pursued,  but  it  might  be  another 
technique.  We  ought  to  have  a fleet  project  team  or  a user  buddy 
for  every  project,  and  let  him  sit  in  on  the  thing  all  the  way. 

I've  got  problems  with  that,  but  on  the  other  hand,  it  is  interesting 
enough  that  we  are  going  to  be  looking  at  it  for  a while  to  see  to 
what  extent  it  may  be  useful. 

Another  thing  we  have  not  yet  started  doing,  partly  because  of  a 
work  load  problem  and  partly  because  I don't  think  we  are  quite 
ready  for  it,  is  what  I'm  calling  end-product  analysis.  This  is 
trying  to  get  at  the  problem  that  technical  reports  are  not  an 
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adequate  document  from  the  user  standpoint  and  are  really  not  the 
end  product  that  is  needed.  Suppose  I have  a subproject  that  is  just 
about  completed  out  of  one  of  our  6.3  projects.  It  has  a little  self- 
administering  task  that  can  be  done  aboard  ship  and  has  a wide  range 
of  possible  application.  Our  problem  is  that  what  we  have  at  the 
moment  is  a technical  report  plus  sets  of  illustrative  materials. 

Is  that  enough  for  us  to  decide  what  to  do  with  it?  The  answer  from 
the  training  manager's  standpoint  is  by  no  means  yes. 

I have  listed  some  of  the  other  kinds  of  things  that  we  as  training 
managers  need  to  know  to  be  able  to  implement  this  thing  and  make  it 
part  of  the  institution.  What  other  applications  of  the  R&D 
findings  or  product  do  we  plan  to  make?  What  is  it  going  to  cost  to 
do  it?  How  much  time  will  it  take  to  print  it?  How  much  to  staff  it? 
What  command  are  we  going  to  assign  to  do  it  (in  this  case  probably 
the  Chief  of  Naval  Education  and  Training  Support,  who  has  a budget 
for  it  or  can  divert  something  else  to  it)?  How  do  we  use  it 
aboard  ship?  Well,  the  resource  requirements  request  is  the  docu- 
mentation we  go  through  in  order  to  get  bucks.  Then  there  is  a 
whole  set  of  implementing  directives  that  will  probably  be  necessary 
and  by  that  I don't  necessarily  mean  what  we  in  the  Navy  call  in- 
structions but  simply  the  papers  that  need  to  get  out  to  make  all 
management  levels  aware  that  this  is  their  responsibility.  It 
may  simply  be  a piece  of  paper;  for  example,  that  says  the  Chief  of 
Naval  Education  and  Training  supports  this,  and  that  your  responsi- 
bilities include  these  things  as  first  priorities.  There  is  a whole 
range  of  those  kinds  of  management  actions. 

I could  go  on  about  getting  these  kinds  of  things  built  into  ISO  models 
and  into  the  costing  models  and  into  Mil  Standard  1379  Alpha  which 
involves  some  kind  of  supporting  analytic  process  so  that  the  con- 
tractor knows  what  to  buy  and  the  kind  of  end  product  that  we  as  users 
need.  What  I am  suggesting  is  that  we  begin  by  trying  to  concentrate 
upon  end-product  analysis.  Let's  start  talking  early  in  the  project 
about  what  kind  of  end  products  are  really  needed  in  order  to 
implement  the  findings  of  this  project,  if  they  come  out  good  enough 
to  be  implementable.  I have  not  laid  this  on  anyone  yet,  so  those 
of  you  who  work  for  me,  either  directly  or  indirectly,  at  some  point 
will  be  asked  to  undertake  some  evaluation  like  this  and  see  whether 
we  can  do  i t or  not  and  see  whether  it  is  useful. 

Now  that  gets  to  another  thing  we  are  trying  to  do  and  that  is  to 
increasingly  build  into  our  projects,  wherever  it  is  appropriate,  an 
applications  planning  phase.  It  is  not  always  appropriate  and  you 
don't  always  do  it  at  the  end.  In  fact  if  you  want  to  talk  .esources 
you  really  need  to  do  it  three  years  ahead  of  time,  as  I have 
indicated.  It  might  be  that  it's  a continuing  process  and  you  do  it 
on  a best  guess  basis  at  the  very  beginning  and  then  refine  it  as  you 
go.  But  we  are  increasingly  trying  to  talk  about  application 
planning.  Again  it's  an  educational  process  for  the  researcher  and 
not  every  researcher  is  adept  at  this  kind  of  thing,  so  it  becomes 
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an  organizational  problem,  a personnel  problem,  and  other  problems 
of  that  type.  On  the  other  hand,  it  opens  the  gate  for  the  re- 
searcher to  pull  in  the  user  training  people  to  talk  about  "What 
if  this  works,  what  is  it  I will  need  to  give  you,  and  how  would 
you  put  it  into  operation?"  So  it  is  another  means  of  bringing 
these  two  parties  together  to  communicate.  And  that  gets  me  to  the 
point  that  many  of  our  projects  require  a management  plan.  Too 
often  management  plans  simply  address  the  principal  commands  who  are 
going  to  be  the  players  in  watching  a thing  grow.  I think  we  need 
to  improve  upon  the  management  plan  and  indicate  what  kinds  of 
products  are  necessary  as  part  of  the  development  process.  That 
is  another  subject  that  is  hanging  up  in  the  air,  and  Bob  Smith  and 
I will  try  to  talk  about  it  and  see  if  we  can  improve  the  management 
plan  part  of  our  projects. 

Another  thing  we  are  doing,  and  this  is  sort  of  a classic  case,  is 
a big  project  on  job  performance  aiding.  We  are  calling  it  test 
evaluation,  even  though  it's  in  6.3.  What  we  are  really  doing 
here  is  taking  the  system  view  of  job  performance  aids.  We  are 
trying  to  take  a big  enough  chunk  out  of  the  Navy  to  find  out  what 
we  have  to  change  in  the  personnel  system  and  the  training  system 
and  the  maintenance  system  in  order  for  the  Navy  to  take  advantage 
of  job  performance  technology  that  already  exists  elsewhere.  We 
are  not  going  to  create  any  more  technology.  We  think  there  are 
all  kinds  of  it  around,  even  though  from  the  research  standpoint 
there  is  a lot  more  that  needs  doing.  What  we  are  trying  to  do  is 
identify  the  change  in  the  total  system  that  will  be  necessary  in 

order  to  help  the  Navy  take  full  advantage  of  this.  Of  course 

there  are  a lot  of  interfaces  and  at  the  present  time  we  are  busy 
developing  strategies  between  us  and  the  users  and  the  research 
people  to  get  the  whole  Navy  user  community  built  into  this  in  the 
right  way  so  that  when  it  comes  down  the  pike  and  is  as  promising 

as  it  looks,  they  will  be  ready  to  implement  it.  It  has  tremendous 

implications  in  terms  of  procurement  of  technical  data  and  in  terms 
of  the  way  people  are  distributed  and  I am  sure  that  I do  not  have 
to  tell  most  of  you  what  the  implications  are.  What  I am  suggesting 
here  is  that  we  become  a little  more  sophisticated  about  planning 
research  and  looking  at  it  in  terms  of  the  total  system,  instead  of 
just  a research  project  with  a technical  report  as  the  product. 

That  gets  me  to  the  point  that  one  of  the  things  we  may  want  to  try 
to  do  when  we  are  a little  more  sophisticated  is  to  take  a project 
and,  as  a pilot  planning  exercise,  start  at  the  capability  end  and 
sort  of  plan  backwards  and  really  lay  out  all  the  steps  tnat  are 
necessary  to  reach  that  capability. 

You  will  note  that  this  differs  from  POM  80  planning,  where  we 
are  talking  about  something  that  is  already  in  process,  something 
we  know  will  be  coming,  something  we  know  to  be  promising,  and 
there  we  are  focusing  around  resource  planning.  Here  I am  talking 
about  the  whole  fusiness  of  user-researcher  interface,  and  how  they 
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have  to  feed  each  other,  and  what  has  to  be  done  all  the  way  along 
to  build  the  implementing  resources.  It  is  vague  at  the  moment, 
but  it  is  hoped  that  at  some  later  time,  when  we  are  more  sophisticated, 
we  will  be  able  to  handle  the  planning  for  some  projects  in  this  way. 

We  in  the  Navy  have  an  experimental  training  policy  board.  We  created 
the  board  a couple  of  years  ago,  and  to  get  a board  to  work  is 
another  bureaucratic  problem.  It  started  out  on  the  basis  that  the 
research  community  needs  to  have  access  to  our  schools  in  order  to 
get  the  advice  of  subject  matter  experts  on  content  and  a variety 
of  other  things,  such  as  space  to  put  the  computer  terminal  in. 

We  had  more  requests  than  we  were  able  to  handle  in  a lot  of  cases, 
so  the  board  was  set  up  to  try  to  prioritize  those  requirements. 

The  role  of  the  board  is  expanding  and  the  board  Is  now  beginning 
to  assume  the  role  of  actually  looking  at  some  projects  to  assess 
their  merits  in  terms  of  user  needs,  cost  versus  likely  payoff,  etc. 
Again,  this  is  a good  opportunity  for  dialogue  between  the  two 
commun i t i es . 

1 think  that  will  necessitate  some  gradual  change  in  the  mix  of 
people  who  are  on  the  board.  What  I find  promising,  given  that  we 
have  had  two  years  to  make  it  work,  is  that  at  least  it  now  is  a 
viable  and  workable  entity.  They  have  problems,  but  at  least  it  is 
working. 

We  have  commented  off  and  on  among  us  that  the  R&D  system  is  not 
designed  for  us.  1 don't  know  if  the  system  is  designed  for  us  or 
not,  but  we  should  learn  to  use  it  anyway.  1 don't  have  much  trouble 
in  making  it  work  if  1 can  get  the  people  to  do  the  things  they 
need  to  do.  1 find  the  bureaucracy  frustrating,  but  also  fascinating 
and  challenging.  The  same  thing  is  true  of  the  documentation  process. 
There  are  all  kinds  of  ways  of  getting  your  stuff  visible--of  getting 
decision  makers  to  look  at  it.  Within  the  Navy  we  have  a capability 
that  the  Chief  of  Naval  Material  operates  for  us.  The  research 
community  regularly  goes  up  the  chain  of  command  through  the  Material 
system.  The  Navy  Personnel  Research  and  Development  Center  is 
commanded  through  the  Chief  of  Naval  Material  organ i za t iona 1 ly , so 
that  is  why  1 address  the  Material  process.  The  opportunity  is 
wide  open  for  the  research  community  to  propose  any  kind  of  training 
or  personnel  systems  that  they  think  could  be  made  to  work.  We 
do  not  see  many  such  proposals.  I don't  know  if  that  is  a problem 
with  the  researchers  with  whom  we  work--that  they  are  not  adept  at 
massaging  the  system  or  at  writing  papers  or  what--but  we  don't  see 
anywhere  near  all  the  system  proposals  that  1,  at  least,  would  like 
to  see.  Maybe  you  think  they  will  be  turned  down,  and  you  don't  like 
that.  I know  that  happens  sometimes  on  the  training  device  side  and 
it  upsets  the  people  who  wrote  the  proposal.  You  don't  always  win. 

I would  certainly  like  to  see  the  researchers  do  a lot  more  thinking 
about  advanced  systems  and  make  those  kinds  of  proposals  to  us.  There 
is  a set  of  documents  called  Science  and  Technology  Objectives  that 


the  Chief  of  Naval  Operations,  through  Bob  Smith,  sends  around.  They 
deal  with  our  area  of  work,  and  I wonder  to  what  extent  the  user 
community  of  the  Navy  Training  Command  has  really  put  a concentrated 
effort  into  getting  those  things  through  the  system.  My  personal 
impression  is  that  within  the  Education  and  Training  Command  in  the 
Navy  there  is  very  little  attention  to  those  documents.  Yet  they 
represent  one  of  the  ways  the  Chief  of  Naval  Operations  communicates 
to  the  Chief  of  Naval  Research  and  to  the  Chief  of  Naval  Material 
about  some  of  his  priority  problems.  They  can  be  used  along  with 
many  other  sources  of  information  by  those  two  principal  R6D  agents 
to  determine  what  the  6.1  and  6.2  programs  ought  to  look  like.  That 
is  our  fault;  we  are  not  communicating. 

Annually  we  do  JRDOD,  which  is  a big  secret  document  (although  our 
piece  of  it  is  not)  that  doesn't  get  reviewed  thoroughly,  and  yet 
it  has  an  impact  on  the  joint  arena  and  on  what  goes  on  in  R6D. 

DoD  periodically  puts  out  Mission  summaries  and  asks  for  comments. 

I have  yet  to  see  any  of  them  come  out  of  the  Training  Command,  al- 
though they  have  a set. 

Hank  Taylor  lately  has  been  running  training  and  personnel  technology 
conferences.  Again,  I do  not  see  much  attendance  there  by  users,  who 
tend  to  think  of  them  as  being  by  researchers  for  researchers. 

Per  dollar,  our  programs  get  more  intensive  review  than  any  other  R&D 
programs  that  exist,  and  yet  we  on  the  user  side  tend  to  look  at  the 
reviews  for  researchers. 

One  other  thing  we  users  do  not  do  is  take  advantage  of  the  capabilities 
within  the  Navy  for  prototype  development.  About  the  only  place  we 
make  prototypes  is  in  the  area  of  training  devices.  It  is  very 
hard  to  get  the  training  community  to  address  itself  to  the  need  for 
prototype  development. 

We  have  continuing  problems  with  the  length  of  Navy  boot  camp.  In 
the  last  ten  years  it  has  ranged  from  7 or  8 weeks  up  to  11  or  12 
weeks.  The  length  is  indeterminable.  We  have  recently  been  driven 
to  put  it  down  to  eight  weeks  and  it  looks  like  OSD  is  talking  about 
seven  weeks  and  somebody  Just  this  last  POM  cycle  said  why  can't  it 
be  six  weeks,  so  I guess  we  will  have  to  answer  that  problem.  In 
going  down  from  nine  weeks  to  eight  weeks  we  had  TAEG  do  a study  of 
boot  camp.  TAEG  did  such  a study  and  to  some  extent  has  redesigned 
boot  camp.  Instead,  why  couldn't  we  have  done  sort  of  a zero  base 
look  at  boot  camp  training  from  the  R&D  standpoi nt--and  that  is  what 
I mean  by  prototype  deve lopment--and  really  set  up  what  it  ought  to 
be.  I'm  not  saying  what  it  would  become,  but  at  least  we  would  have 
a boot  camp  that  had  some  additional  rationality  to  its  content  and 
approach. 

In  summary,  I have  talked  about  different  definitions  of  products, 
about  several  dimensions  of  the  need  for  better  processes  of  planning 
for  application,  about  the  user  doing  a bettei  job  of  trying  to 
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address  what  he  is  getting  and  what  he  needs,  and  about  the  need  for 
more  attention  both  to  proposals  and  requirements  within  the  "systems" 
context . 


Air.  Melvin  A.  Schwartz 
Personnel  Research  Psychologist 
Navy  Personnel  R&D  Center 


The  case  history  we  are  going  to  discuss  today  is  part  of  the  Shipboard 
Manning  and  Automation  Program  which  is  being  conducted  out  at  David 
Taylor  Naval  Ship  Research  and  Development  Center  in  Annapolis.  The  title 
of  the  paper  on  which  my  remarks  are  based  is  called,  "The  Utilization 
of  People-Related  Research  and  Development;  A Case  History  of  the 
Shipboard  Facilities  Maintenance  Demonstration  Study." 

In  my  talk  today,  I am  going  to  cover  four  basic  areas: 

1.  An  historical  background  of  the  program. 

2.  A brief  description  of  a study  that  we  conducted  and  are  in  fact 
still  conduct i ng . 

3.  A brief  discussion  of  the  problems  that  we  experienced  in  con- 
ducting the  initial  phases  of  the  research,  and  also  the 
problems  which  have  implications  for  the  later  utilization  and 
acceptance  of  results. 

h.  We  will  end  with  a parochial  review  of  positive  and  negative 
factors  which  we  believe  to  affect  the  utilization  of  results. 

Back  in  1972,  in  light  of  the  cost  of  personnel  and  the  shortage  of 
manpower  due  to  the  all  volunteer  force  structure.  Admiral  Zumwalt, 
then  CNO,  issued  an  action  memorandum  directing  that  laboratories 
and  various  Navy  organizations  become  concerned  with  reducing  manpower 
in  the  fleet.  A committee  was  appointed,  and  a joint  fleet  laboratory 
program  was  established,  with  DTNSRC  designated  the  lead  laboratory. 
Participants  in  the  program  included  the  Navy  Personnel  Research  and 
Development  Center  Destroyer  Development  Group,  and  we  had  inputs  from 
the  Navy  Ship  Engineering  Center  and  from  the  Chief  of  Naval  Material 
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Combat  System  Advisory  Group.  The  objective  of  the  total  program  was 
to  reduce  manning  while  maintaining  or  improving  operational  effec- 
tiveness and  shipboard  readiness. 

The  general  approach  was  to  conduct  a set  of  pilot  studies.  We 
started  off  by  doing  a pilot  study  on  reduced  bridge  manning,  where 
we  introduced  all  kinds  of  new  equipment,  procedures,  work  space 
layouts,  and  manning  alternatives  to  see  if  we  could  reduce  the  number 
of  people  on  the  bridge.  The  study  was  conducted  using  both  mockups 
and  quite  a few  operational  ships.  It  was  conducted  with  the  assistance 
of  the  Destroyer  Development  Group.  Ships  were  deployed  with  alter- 
native configurations  of  equipment  sets,  which  were  designed  to  save 
manhours,  and  we  designed  alternate  procedures. 

The  results  of  the  study  were  generally  positive.  We  found  that  by 
automating  certain  equipment  and  functions  we  could  reduce  the  number 
of  men  standing  watch  on  a destroyer  class  ship  or  an  FF  1052  class 
ship  from  12  people  in  the  pilot  house  and  related  areas  to  about  5* 

When  the  results  of  that  study  were  initially  disseminated,  many 
operators  expressed  considerable  resistance  to  the  implications  of 
those  findings.  A typical  response  was,  "If  what  you  say  is  true  and 
you  can  reduce,  say,  seven  people  times  three  watch  stations,  you 
are  going  to  take  21  people  off  my  ship,  and  I'm  already  having 
trouble  getting  done  my  damage  control,  my  facilities  maintenance,  and 
a bunch  of  other  shipboard  functions." 

Upon  hearing  these  objections  raised  by  operators,  the  scope  of  our 
program  was  expanded  to  include  many  other  functional  areas. 

Facilities  maintenance  was  one  of  them.  It  is  sort  of  punching  into 
a big  bowl  of  jelly.  You  have  solved  bridge  manning  problems  and 
out  pops  a damage  control  problem  or  facilities  maintenance  problem. 

The  facilities  maintenance  area  was  assigned  to  me,  and  I have  become 
known  as  "Mr.  Clean"  for  the  Navy.  It  is  a study,  and  it  does  respond 
to  a user  need.  Facilities  maintenance  includes  those  actions  per- 
formed by  ship's  force  necessary  to  maintain  conditioning,  cleanli- 
ness and  appearance  of  exterior  and  interior  ship  spaces,  hull  and 
hull  fittings,  but  not  including  below  the  water  line.  To  be  more 
specific,  it  includes  painting,  chipping,  scraping,  peeling,  waxing, 
washing,  buffing  and  all  those  tasks  that  your  wives  do  at  home  or 
your  husband  or  boyfriends  or  whatever.  We  did  not  do  an  extensive 
cost-benefits  analysis,  but  upon  trying  to  get  a handle  on  how  big 
the  manpower  expenditure  problem  was,  we  looked  at  the  average  annual 
personnel  costs  from  a manpower  costing  document  and  used  as  a 
hypothetical  population  125  FF1052  class  ships. 

We  found  that  they  are  spending  roughly  $2A0,000  per  ship  per  year  in 
just  manhour  dollars  for  getting  that  work  done.  That  comes  to  an 
equivalent  of  27  men  on  a ship  manned  by  over  200  people,  working 
full  time  on  nothing  but  these  tasks.  That  is  not  to  say  that  they 
do  only  that,  since  the  way  the  functions  are  currently  administered 
aboard  ship  they  will  grab  whoever  is  available,  usually  the  ncn- 
rated  seaman.  Life  cycle  costing  for  that  same  population  of  125 


destroyers  conservatively  is  in  excess  of  3 billion  dollars.  If 
there  were  only  125  ships  in  the  fleet  we  could  possibly  live  with 
that  life-cycle  cost,  but  there  are  well  over  kOO  ships  in  the  fleet. 


An  initial  analysis  of  the  problem  in  the  manpower  area  indicated 
there  were  many  impromptu  organizations  established  to  do  it. 
Departments  had  their  own  areas  of  responsibility  for  cleaning. 

It  was  not  a systematic  approach  to  the  problem.  The  people  doing 
the  jobs  did  not  have  adequate  training.  You  may  wonder  what  training 
you  need  to  swing  a swab  or  vacuum  a floor.  You  do  need  training. 
Certainly  they  were  not  motivated  to  do  that  kind  of  work.  They 
joined  the  Navy  because  they  saw  a T.V.  flick  with  a guy  with  earphones 
on  his  head  watching  a scope.  Swinging  a swab  was  not  their  idea  of 
a career  in  the  Navy.  There  were  not  standard  work  procedures  and 
management  techniques,  there  were  no  systematic  work  schedules.  Now 
the  Navy  has  developed  a thing  called  the  PMS,  the  Planned  Maintenance 
System,  which  has  been  used  to  some  degree  with  success. 

Through  the  Planned  Maintenance  System,  maintenance  of  operational 
equipment  such  as  radar,  sonars,  motors,  gun  shooting  things  and 
propulsion  things  have  been  addressed  in  a systematic  fashion.  We 
now  have  routine  and  periodic  procedures  for  maintaining  them.  We 
also  have  an  extensive  reporting  system,  yet  there  is  nothing  like 
that  for  facilities  maintenance  in  the  fleet  today.  The  work  pro- 
cedures that  they  use  are  inappropriate,  or  outdated.  A lot  of 
cosmetic  painting  goes  on.  People  don't  know  the  difference  between 
rust  and  red  lead.  People  don't  know  that  it  is  not  right  to  put 
paint  on  without  washing  the  surface.  People  don't  know  that  some- 
times you  do  not  need  to  paint,  but  could  just  wash  the  surface. 

Equipment  and  materials,  standard  Navy  issue  items,  has  not  seemed 
to  keep  pace  with  modern  technology  in  the  janitorial  services 
business.  The  Royal  Navy  has  come  to  grips  with  the  problem  and 
so  has  our  hotel  industry  and  all  our  janitorial  maintenance 
facilities  all  over  this  country  in  the  civilian  sector,  yet  nobody 
in  the  Navy  heretofore  has  ever  addressed  this  problem  from  the 
same  standpoint,  at  least  not  as  one  problem. 

Especially  on  older  ships,  the  spaces  are  simply  not  designed  for 
maintainability.  It  is  hard  to  clean  mazes  of  overhead  wi reways  and 
do  a deck  that  has  so  many  beams  coming  out  of  it,  and  no  false  bulk- 
heads in  it.  The  problem  of  maintainability  is  clear  to  the  Navy  now, 
and  people  are  addressing  the  problem  of  redesign  for  ease  of  main- 
tenance. The  cumulative  result  of  all  of  these  problems  put  together 
is  that  manpower  is  significantly  wasted,  material  is  wasted,  ships 
deteriorate,  and  there  is  a lowered  morale  and  work  motivation,  not 
only  for  the  people  doing  the  jobs  but  for  the  people  living  and 
working  in  the  spaces  that  those  people  are  supposed  to  be  doing  their 
jobs  in. 

These  problems  have  been  hanging  around  for  at  least  a hundred  years. 
Part  of  the  problem  is  due  to  the  fact  that  nobody  ever  addressed 
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them  all  at  once.  Who  cares  if  the  use  of  disposable  mess  gear  means 
that  dishes  don't  have  to  be  washed .. .when  you  compare  that  with  the 
need  for  a new  ship  in  the  Navy.  If  you  pulled  together  the  entire 
maintenance  job,  you  might  have  a different  perspective.  Another 
reason  for  the  inaction  is  that  operational  criteria  have  never  been 
defined  in  the  maintenance  arena.  How  clean  is  clean,  or  how  corroded 
is  corroded,  depends  on  who  is  viewing  the  situation.  Even  if 
appropriate  criteria  were  developed  and  addressed,  the  resources  have 
never  been  pulled  together  to  conduct  a longitudinal  study.  And 
that  is  part  of  the  bureaucratic  problem. 

Lastly,  the  Navy  has  a "can  do"  spirit,  and  they  do  remarkably  well 
with  it.  Tradition  has  been  one  of  the  major  obstacles  to  getting 
the  facilities  maintenance  problems  addressed.  The  Navy  had  the 
manpower  to  do  these  jobs  and  in  certain  situations  they  were  not 
required  to  man  the  guns  or  man  watch  stations  and  so  they  cleaned 
the  ship.  But  the  situation  has  changed.  There  is  no  reason  to 
suppose  that  the  problems  are  going  to  go  away  either.  The  design 
trends  in  new  ships  indicate  that  there  is  going  to  be  increased 
sophistication  in  automation.  That  means  decreased  crew  size  as 
per  the  bridge  manning  studies.  But  at  the  same  time,  ships  are 
being  increased  in  size,  and  there  is  a decrease  in  the  non-rated 
watch  standees  that  are  going  to  be  available  for  this  kind  of  work. 
Personnel  problems  have  increased  since  we  have  an  a 1 1 -vol unteer 
force  structure.  People  have  a higher  expectation  of  quality  of 
life  and  they  are  going  to  express  their  objections  to  facilities 
ma i ntenance . 

The  objective  of  our  study  was  to  demonstrate  a reduction  in 
facilities  maintenance  (FM)  manhour  expenditures  while  improving 
the  condition,  cleanliness  and  appearance  of  shipboard  spaces.  The 
approach  we  took  in  our  first  phase  of  the  study  was  to  analyze  the 
facilities  maintenance  problems  and  future  facilities  maintenance 
requirements.  We  did  a space  analysis  of  about  30  or  AO 
percent  of  the  shipboard  spaces  aboard  a destroyer  and  determined 
what  the  cleaning  and  surface  preparation  and  corrosion  control 
requirements  really  were.  We  generated  candidate  concepts  in  each 
of  three  or  four  concept  areas,  s ch  as  manpower,  equipment  and 
materials,  and  training.  We  screened  the  various  labor-saving 
devices,  materials  and  procedures,  and  developed  physical  repre- 
sentation of  these.  It  was  not  that  we  wanted  to  bring  aboard  a 
particular  piece  of  equipment  to  solve  a problem.  We  wanted  to 
know  if  the  idea  of  using  that  piece  of  equipment  would  help  in 
reducing  the  manhour  requirements  in  getting  the  jobs  done  more 
efficiently. 

We  designed  an  experiment  and  selected  variables  of  interest 
including  cleanliness  and  appearance  of  the  spaces.  We  developed 
a rating  scale  and  a skill  and  knowledge  test  to  determine  the 
knowl edgeab i 1 i ty  of  people  doing  the  jobs.  We  used  an  attitude  and 
motivation  questionnaire,  and  a number  of  other  briefing  and 
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debriefing  questionnaires  which  got  at  the  shipboard  personnel  attitudes. 
We  installed  the  physical  representations  aboard  an  operational  ship, 
deployed  the  ship  for  about  a year,  analyzed  the  data,  and  reported 
our  findings.  In  short,  we  wanted  to  determine  empirically  whether 
innovations  in  each  of  our  concept  areas  contributed  something  of 
value  to  manpower  requirements,  cleanliness  and  appearance  levels, 
attitudes  of  the  crew,  and  crew  skill  and  knowledge.  We  also  wanted 
to  determine,  within  each  concept  area,  which  of  the  concepts  seemed 
to  hold  the  biggest  bang  for  the  buck,  and  also  later  on  in  the 
program  to  determine  the  optimally  effective  mix  of  concepts  and 
specifics  (specific  equipment,  materials,  procedures,  and  training 
systems) . 

Now  I would  like  to  talk  about  the  results  of  our  Phase  I.  We  had 
many  problems  in  conducting  fleet  research.  It  was  a sheltered 
field  study;  we  had  the  ship  on  a not-to-interfere  basis;  even  boarding 
the  ship  was  a problem,  but  we  will  get  to  some  of  these  problems 
later.  The  results,  in  fact,  indicated  that  approximately  ^2  percent 
of  the  manhours  can  be  saved  in  comparison  with  documented  manhour 
expenditures  and  ship  manning  documents  for  the  class.  And  that  does 
not  include  surface  preparation  and  painting,  which  we  did  not 
address.  We  set  up  an  eight  man  team  to  do  virtually  all  of  the 
facilities  maintenance  tasks  for  all  common  use  spaces  plus  the  normal 
spaces  assigned  to  the  first  division.  More  spaces  were  assigned 
per  person,  yet  the  cleanliness  and  appearance  rating  forms  indicated 
that  there  was  a decided  improvement  in  cleanliness  and  appearance. 

The  ski  1 1 and  knowledge  of  members  of  the  team,  as  measured  by  our 
ad  hoc  skill  and  knowledge  test,  increased  very  significantly  both 
in  a statistical  and  practical  sense.  There  was  no  apparent  effect 
on  morale  and  motivation  due  to  the  FM  innovations. 

One  of  the  limitations  in  our  research  was  that  only  40  percent  of 
the  ships'  spaces  were  involved  and  we  did  not  really  have  the  total 
system  involved.  We  did  not  consider  a function  which  takes  up  about 
25  percent  of  all  the  facilities  maintenance  manhours;  i.e.,  surface 
preparation,  corrosion  control  and  painting.  We  did  not  do  any  bilge 
cleaning,  and  only  first  division  personnel  were  involved  in  the 
study.  During  the  research  implementation  phase  we  encountered  many 
problems  which  gave  us  heartburn  in  collecting  the  data  and  certainly 
would  interfere  with  later  acceptance  and  utilization  of  any  of  the 
positive  findings  of  the  program. 

There  was  organizational  resistance  to  change.  Some  people  believe 
that  you  don't  really  need  new  material  and  equipment  or  training 
programs.  They  believe  that  leadership  and  accountability  will  take 
care  of  all  of  your  problems.  I think  this  has  been  demonstrated 
not  to  be  the  case,  that  you  do  need  more  than  accountability.  We 
had  difficulties  in  boarding  the  ship,  which  was  deployed  all  over 
South  Africa.  We  had  difficulty  in  getting  out  the  reports.  We 
felt  that  it  was  best  to  communicate  results  to  operators  and  users 
in  their  own  language  by  their  own  people.  However,  the  Destroyer 


-188- 


r 

Development  Group,  which  is  part  of  this  program,  consisted  of  only 
an  operator  and  a consumer,  and  because  of  rotation  of  personnel 
through  the  office,  it  took,  quite  a while  for  them  to  publish  their 
study.  Well,  operators  are  less  likely  to  read  a laboratory  report 
than  they  are  to  read  a report  from  another  operator.  I don't 
really  like  the  demarcation,  but  nevertheless  it  does  exist. 

In  continuing  the  research  effort,  there  was  threat  of  a funding 
cut.  We  thought  we  were  going  to  get  cut  off  without  ever  being 
able  to  get  into  Phase  II.  We  did  not  have  a requirements  document, 
and  this  came  back  to  haunt  us  later  on  when  the  funding  was  taken 
away  temporarily.  We  went  looking  for  sponsors  to  continue  the 
research  program.  Sponsor  responsibilities  for  continuing  the  re- 
search are  fractionated  because  there  are  equipments  involved  that 
have  to  be  service-approved,  and  that  is  done  by  the  Naval  Ships 
Engineering  Command.  I'm  sure  you're  familiar  with  the  problem  of 
fractionation  of  responsibilities. 

Despite  the  problems,  we  finally  got  our  money  and  are  now  continuing 
into  Phase  II.  I would  like  not  to  make  a value  judgment  at  this 
point  in  our  work,  except  to  say  that  i think  some  of  the  results 
are  being  accepted  prematurely  when  we  don't  have  adequate  evidence. 
The  FFG  7 is  a brand  new  ship  being  built  in  Bath,  Maine.  Their 
manpower  man  used  to  work  on  the  project  with  me.  He  helped  develop 
some  of  the  innovations.  In  fact,  during  the  entire  study  and 
implementation  p.ogram,  we  got  the  whole  fleet  involved.  We  asked 
them  to  help  select  the  equipment,  procedures  and  materials  all  the 
way  through.  In  the  FFG  7 project  office,  there  is  now  a guy  who 
is  to  work  the  project  with  us.  One  ship  is  putting  our  whole 
project  on  their  ship  and  in  turn  will  give  us  data  on  how  well  the 
innovations  work. 

PMS306,  a newly  established  office  for  fleet  maintenance,  became 
very  interested  and  asked  if  there  was  anything  they  could  help  us 
with.  We  brief  them  and  keep  them  informed  of  all  the  developments 
in  the  program,  and  perhaps  sometime  in  the  future  they  will  become 
very  involved.  CINCPACFLT  read  the  Destroyer  Development  Group 
report  and  became  interested.  They  wanted  to  do  it  right  now,  and 
again  we  suggested  they  wait  until  we  finish  our  research.  It 
will  be  two  or  three  years  till  we  get  all  the  data  in  and  analyzed, 
so  they  decided  to  do  a study  just  like  ours. 

Going  into  Phase  II  we  want  to  do  the  same  thing  on  an  expanded 
level.  We  want  to  develop  new  procedures  for  an  entire  ship  class, 
actually  more  than  one  ship  class,  this  time.  We  like  to  believe 
that  the  concepts  that  we  are  evaluating  will  be  just  as  valid  for 
an  LST  or  an  aircraft  carrier  as  for  the  destroyer  class  ships. 

We  are  also  developing  a large  management  information  system, 
where  the  contractor  will  go  in  and  analyze  facilities  ''lainter 
requi  rements  and  develop  a work  management  distribution  s-.  ' 
very  much  like  the  one  we  used  in  the  first  phase  of  t'-  ' 
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We  will  also  be  preparing  technical  manuals.  I failed  to  mention 
that  we  have  also  developed  13  audiovisual  training  modules,  things 
like  how  to  strip  a deck,  how  to  clean  a head,  safety  in  facilities 
maintenance,  the  use  of  chemicals,  etc.,  and  we  feel  that  this  is 
at  least  partly  responsible  for  the  improvement  in  skill  and  know- 
ledge as  well  as  in  the  cleanliness  and  appearance  of  the  shipboard 
spaces . 

We  are  also  going  to  develop  more  modules  in  the  area  of  corrosion 
control  and  surface  preparation.  We  are  always  looking  at  labor-saving 
devices  and  equipment  and  materials,  and  we  are  tooling  up  now  to 
purchase  those  for  the  test-ship  population  in  the  next  phase  of 
our  study.  We  are  going  to  do  an  entire  ship,  and  consider  all 
the  interactions  among  the  various  department  and  divisions  aboard 
ship.  And  we  plan  to  do  a cost-benefit  trade-off  analysis  to  show 
where  the  biggest  bang  was.  The  approach  will  be  essentially  the 
same  as  it  was  in  Phase  I. 

In  terms  of  a retrospective  analysis,  and  again  I have  to  think  it's 
a little  parochial,  but  if  we  had  to  do  it  over  again  we  would  look 
in  the  R&D  planning  phase  for  the  following  features: 

1.  Ensure  widespread  need  recognition,  and  we  really  never 
had  any  problem  with  this. 

2.  Establish  ad  hoc  teams  of  potential  sponsors,  users,  study 
participants  and  researchers.  The  team  responsibilities 
would  include  the  generation,  coordination  and  dissemination 
of  a specific  operational  requirement  document,  something  that 
would  give  us  a charter  and  license  to  operate,  backed  up 
with  money.  The  team  would  also  establish  the  study  objective 
jointly,  each  participating  at  his  own  level  of  capability  and 
in  his  own  area  of  concern.  But  everyone's  views  should  be 
jointly  considered. 

3.  Develop  the  study  approach.  The  researcher  cannot  afford  to 
remain  in  an  ivory  tower;  he  has  to  deal  with  the  realities 
of  collecting  data  aboard  a ship  that  is  floating  around  in 
the  ocean,  and  also  get  the  plan  of  logistic  support  for  the 
study . 

Now,  the  planning  for  the  logistic  support  of  the  study  bears  little 
relationship  to  the  ultimate  logistic  support  that  will  be  required 
for  utilization  of  the  system,  and  both  should  be  addressed.  The 
support  requirements  must  be  considered  in  early  planning,  we  feel, 
in  order  to  ensure  utilization  later  on.  You  should  acquire  a 
competent  and  motivated  research  staff  and  adequate  facilities. 

We  have  been  fortunate  in  this  respect.  You  should  jointly  develop 
the  study  design,  the  innovations  to  be  evaluated,  and  the  test 
procedures  and  schedules,  and  seek  change  advocates  at  all  levels 
to  avoid  the  not- i nvented-here  syndrome.  I think  we  have  had  a 
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reasonable  degree  of  success,  although  we  did  have  problems.  There 
were  one  or  two  people,  some  on  the  facilities  maintenance  team  and 
some  at  team-manager  level,  that  felt  accountability  was  the  only 
answer.  This  really  impeded  progress  for  a while,  but  there  was 
one  guy  on  the  team  who  really  ate  this  stuff  up.  He  got  into 
doing  the  job,  showing  people  on  the  team  how  to  do  the  Job.  That, 
coupled  with  the  fact  that  these  people  did  not  have  to  stand 
watches  and  only  had  to  work  half  a day,  helped  them  to  tolerate 
this  program  for  a considerable  length  of  time  until  it  got  to  be 
too  much.  Three  months  of  it  is  about  all  you  can  have.  And  that 
has  implications  for  the  design  of  our  next  team  concept.  Maybe 
we  should  consider  a permanent  team,  a rotatable  pool  of  people, 
as  they  do  in  mess  cooking  in  the  fleet  today.  We  are  going  to  play 
with  that  concept  in  this  next  phase. 

You  must  begin  with  the  early  research  planning  phase.  Even  though 
you  do  not  know  anything  about  how  the  innovations  are  going  to  work 
out,  you  have  to  begin  to  plan  your  transition  requirements. 

In  Phase  B,  development  and  testing,  inputs  from  the  users  and 
industry  and  system  development  people  should  be  considered.  Joint 
installation  of  systems  and  joint  data  collection  effort  should  be 
arranged  wherever  practicable.  The  study  should  use  the  people  in 
the  fleet  if  they  have  the  time  and  are  willing  to  cooperate.  If 
they  are  part  of  the  program,  you  will  have  more  success  in  com- 
pleting the  study,  and  if  the  results  are  positive,  you  will  find 
it  a lot  easier  to  put  the  systems  aboard  ship. 

You  have  to  be  prepared  to  compromise  your  experimental  design.  You 
have  to  keep  the  sponsors,  users  and  potential  transition  organization 
advised  of  progress  and  intermediate  developments  and  findings. 
Audiovisual  programs  can  be  very  helpful.  The  researcher  should 
follow  through  with  organizations  to  be  involved  in  the  transition. 

We  should  specify  the  requirements  for  utilizing  the  research  results 
in  as  much  detail  as  practicable.  Include  requirements  for  approval 
of  logistic  support  requirements,  and  requirements  for  additional 
studies,  such  as  studies  of  reliability  of  equipment  and  study  of 
safety  and  costs. 

Finally,  an  essential  step  is  communication  of  your  study  findings. 
Researchers,  sponsors  and  users  should  participate  in  the  planning 
for  fleet  implementation.  When  you  are  reporting  your  study  findings, 
it  helps  to  target  your  communications  at  relatively  homogeneous 
audiences.  The  development  agencies,  and  these  are  all  the  commands 
that  are  concerned  with  training  and  equipment  utilization,  logistics, 
and  safety,  should  consult  together  to  develop  a follow-up  plan  for 
utilizing  the  results.  You  may  want  to  read  our  document  entitled 
Facilities  Maintenance  Demonstration  Study  for  a better  understanding 
of  the  study. 

I think  that's  all  I have  to  say. 
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COMMENT  FROM  THE  AUDIENCE;  I'd  like  to  comment  on  your  heavy  emphasis 
on  coordinating  and  planning.  I get  concerned  that  maybe  sometimes 
we  only  add  to  the  length  of  the  research  and  thereby  confuse  the 
issue  even  more  when  we  over-extend  this  coordination.  I am  in  fui 1 
agreement  with  you  that  we  ought  to  have  the  appropriate  people  involved, 
but  I had  a feeling  here  that  we  were  going  in  the  direction  of  overkili. 

MR.  SCHWARTZ:  You  have  to  understand  from  my  own  point  of  view,  that 
this  was  an  exciting  probiem  to  work  on  in  that  the  solutions  were 
so  crystal  clear  and  nobody  could  really  argue  with  what  the  approach 
was.  It  was  easy  to  convince  people  that  they  had  a problem  and 
they  needed  a reasonable  attack  on  it.  I admit,  however,  that  the 
more  you  get  into  committee  planning  efforts,  the  more  lengthy  and 
complex  the  whole  approach  may  become  and  that  will  extend  your  re- 
search time.  That  may  even  be  what  keeps  you  from  doing  it. 

COMMENT:  The  trade-off  here  seems  iike  a good  one.  That  is  to  say, 
people  don't  like  to  ciean  ships;  it's  not  something  you  wi i 1 get  a lot 
of  support  for;  it's  a chore.  I doubt  that  this  appiication  has  been 
overdone,  and  that  is  a virtue.  And  this  retrospective  plan  that  you 
have,  while  comprehensive,  has  a better  chance  of  achieving  results 
than  if  you  did  not  get  people  involved. 

DR.  SANDS:  Are  there  any  other  questions  on  either  paper? 

QUESTION:  A quick  question  on  the  first  paper.  Did  you  ever 
define  products?  It  would  seem  to  me  appropriate  for  your  purposes 
to  make  clear  how  your  definition  differs  or  relates  to  the  way  the 
GAO  defines  products. 

MR.  MALEHORN:  My  understanding  is  that  the  GAO  defines  a product  in 
terms  of  technical  reports.  We  would  say  that  it  really  depends  upon 
the  project  itself  and  what  its  implications  are.  I'm  not  sure  I 
could  even  say  there  is  any  minimum  standard.  When  we  were  trying 
to  develop  a response  to  the  GAO  report,  we  spent  a couple  of  hours 
talking  about  units  of  measurement.  Otherwise  you  don't  have  any 
type  of  standard.  So,  I think  a product,  from  the  standpoint  of 
impiementation,  could  be  a whole  range  of  things  and  it  just  depends 
on  the  particuiar  project  and  what  its  implications  are.  The  issue 
of  what  is  a product  really  is  kind  of  critical  for  us  as  a community. 

We  say  "utilization"  and  "research  utiiization"  when  we  really  mean 
product  utilization.  So  you  must  be  very  clear  on  what  product  means 
in  order  to  develop  a viable  strategy  and  system  that  the  Congress 
and  everybody  else  can  understand  and  accept. 

COMMENT:  I don't  understand.  if  you  knew  that  a report  of  the  findings 

of  a study  were  to  be  its  product,  and  you  wanted  to  get  those  findings 
utiiized,  wouldn't  you  refer  to  that  as  research  utilization? 
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MR.  MALEHORN:  Well,  if  the  report  is  a product,  then  you  go  a very 
different  way;  you  get  a report  on  whether  or  not  you  had  report 
utilization.  If,  on  the  other  hand,  it  happens  to  be  an  instructional 
unit,  you  have  a different  criterion  for  utilization.  It  sounds 
trivia?  at  first,  but  it  is  not.  I think  you  want  to  prepare  a list 
which  includes  various  possible  types  of  contributions. 


-193- 


POLICY  RECOMMENDATIONS  FOR  IMPROVED  RESEARCH  UTILIZATION 


SUrWARY 


For  purposes  of  this  session  smatl  groups  met  by  militax^  branch 
to  develop  policy  recommendations  for  improved  utilization  of  human 
resources  RDT&E^  and  then  reported  their  recormendations  in  plenary 
session. 


The  Air  Force  group^  chaired  by  COL  Ralph  Hoggatt,  Air  Force  Human 
Resources  Laboratory ^ made  the  following  recommendations: 

, A research  utilization  system  should  be  formal  and  should  be 
documented. 

. Effective  RU  requires  an  intimate  and  continuing  process  of 
interaction  between  user  and  producer. 

. The  "contract"  between  researchers  and  users  should  be  compre- 
hensive and  definitive,  specifying  such  things  as  objectives  of 
the  research,  products,  uses,  users,  logistics  requirements, 
cost  estimates,  and  risks.  i 

. There  should  be  an  ad  hoc  corrmittee  to  monitor  the  R&D  project. 

. Cost-benefit  analysis  is  essential. 

. The  applications  function  is  a full-time  job. 

. There  must  be  high-level  endorsement  of  findings  and  recommen- 
dations. 

. A plan  for  marketing  the  product  is  essential. 

. There  must  be  evaluation  of  the  product  by  the  user  as  well  as 
research  community. 

Judah  Drob,  Department  of  Labor,  recommended  that  the  conferees  urge 
DoD  to  appoint  someone  at  a high  level  to  be  responsible  for:  stimu- 
lating action  by  each  of  the  services  in  the  area  of  utilization, 
promoting  interchange  of  RU  experiences  among  the  services,  and 
encouraging  the  institutionalization  of  RU.  He  suggested  that  such 
an  appointee  should  be  asked  to: 

. Obtain  appointment  of  counterparts  in  the  services. 

. Convene  conferences  to  promote  utilization. 

. Urge  assignment  of  utilization  specialists  in  research  and 
user  organizations. 

. Organize  linking  units. 

. Develop  storage  and  retrieval  capabilities. 

. Promote  understanding  of  RDT&E  and  research  utilization. 

Dr.  Arthur  Drucker,  Army  Research  Institute,  reported  the  following 
recommendations  by  the  Army  group: 

. There  is  need  for  a coherent  system  for  accumulating  and  priori- 
tizing research  requirements. 
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. Such  ppioritizing  should  not  be  the  funation  of  the  researohev , 

. All  potential  consumers  should  be  involved  early. 

. The  research  program  should  be  responsive  primarily  to  guidance 
documents . 

. There  is  need  for  users  to  develop  an  understanding  of  the  research 
process  and  of  what  can  be  expected  by  way  of  research  products. 

Bert  Griffist  Rehabilitation  Services  Administration , offered  a set 
of  whimsical  prescriptions  for  improved  utilization. 

Dr.  Robert  Smithy  Office  of  the  Chief  of  Naval  Operations ^ reporting 
for  the  Navy  group ^ suggested  that  the  Navy  human  resources  RDT&E  pro- 
gram is  both  relevant  and  cost-effective  He  reported  the  following 
recommendations  for  becoming  even  more  effective  in  utilization: 

. There  is  need  for  early  and  continual  joint  planning  for  utilization, 
including  identification  of  people  to  be  kept  informed,  generation 
of  cost-benefit  data,  identification  of  end  products  and  specifi- 
cation of  implementation  requirements. 

. Resources  for  utilization  should  be  increased. 

. Application  officers  should  be  assigned  to  key  user  agencies. 

. The  R&D  corrmmity  should  provide  technical  assistance  to  the 
user  community. 

. Technical  reports  should  have  executive  summaries  that  contain 
cost-benefit  information  and  indicate  eo:pected  users  and  uses 
of  the  report. 

• There  should  be  an  annual  report  of  utilization. 

. There  should  be  a system  for  tracking  utilization. 

In  the  ensuing  discussion  Dr.  Rostker  expressed  dissatisfaction  with 
the  recommendations,  suggesting  that  they  were  largely  a recapitula- 
tion of  existing  procedures.  Dr.  Vhlaner  cormented  that  the  issues 
we  are  struggling  with  are  long-standing  and  complex,  and  not  likely  to 
be  amenable  to  standardized  solutions . He  further  suggested  that  progress 
will  be  largely  a funation  of  the  skill  and  dedication  of  people,  and 
that  the  military  system  is  well  endowed  with  this  resource  on  both 
the  user  and  researcher  side.  CART  Clarkin  emphasized  the  need  for 
a lessening  of  the  isolation  of  the  respective  communities,  while 
maintaining  a responsible  autonomy. 

Dr.  Sands  concluded  with  the  hope  that  the  symposium  will  have  con- 
tributed to  the  dedication  of  people  to  build  the  kinds  of  linkages 
that  have  been  suggested  throughout  the  conference  in  the  interest 
of  improved  utilization  of  people -related  RDTSE. 


POLICY  RECOmENDATIONS  FOR  IMPROVED  RESEARCH  UTILIZATION 


THE  CHARGE  TO  THE  CONFEREES 

DR.  SANDS:  I would  like  to  talk  about  what  it  is  we  intend  to  do  for 
the  remainder  of  the  day,  and  what  it  is  we  are  generally  expecting 
for  tomorrow.  As  was  indicated  earlier,  Dr.  Rostker  will  be  attending 
the  8:00  a.m.  meeting  and  he  will  remain  through  the  morning  session. 

He  is  doing  that  for  the  purpose  of  providing  opportunity  to  continue 
the  dialogue,  and  I'm  certain  from  talking  with  him  that  he  wants  to 
encourage  dialogue. 

With  reference  to  the  15^5  group  meetings,  we  are  going  to  ask  certain 
members  of  the  faculty  to  sit  in  with  the  Army  and  the  Air  Force  since 
these  groups  are  rather  small. 

I think  we  have  come  to  a point  where  I would  like  to  reflect  on  what 
it  i s we  have  done  this  week.  First  of  all.  Admiral  Metcalf  high- 
lighted many  of  the  issues  in  research  utilization.  These  issues 
were  problem-oriented.  That  discussion  was  followed  by  an  assessment 
of  our  track  record,  so  to  speak,  based  primarily  on  the  view  from 
GAO  and  the  interpretation,  documentation  and  reply  by  DDR6E  as 
reviewed  by  Colonel  Taylor. 

Then,  at  the  banquet  on  the  evening  of  the  first  day,  we  had 
Dr.  Rostker's  address,  originally  intended  to  be  the  keynote  address, 
which  we  have  revisited  today.  Dr.  Rostker,  as  I see  it,  was 
attempting  to  underline  the  current  expectations  for  people-related 
RDT&E.  Are  these  expectations  being  fulfilled?  And  if  they  are  not 
being  fulfilled,  what  appears  to  be  over  the  horizon?  What  are  some 
of  the  major  thrusts  that  may  impact  on  the  management  of  RDT&E  in 
the  near  future? 

We  also  had  a state-of-the-art  session,  where  we  heard  and  discussed 
perspectives  from  a number  of  federal  agencies  and  other  institutions; 
and  there  was  an  attempt  to  relate  those  experiences  to  the  literature. 
We  met  in  our  respective  military  service  groups  for  purposes  of 
certifying  many  of  the  shortfalls,  needs,  problems  and  issues  that 
had  been  mentioned  by  Admiral  Metcalf  and  Colonel  Taylor.  Today  we 
considered  a number  of  case  problems,  with  an  attempt  to  identify 
specific  utilization  strategies  that  might  be  applied  to  improve 
the  use  of  the  particular  innovative  products  described  in  those 
studies.  This  afternoon  we  heard,  from  Merle  Malehorn,  what  seemed 
to  have  been  a process  methodology  that  appears  to  make  a lot  of 
sense  to  me,  followed  by  a case  study  portraying  a number  of  success- 
ful instances  of  utilization  strategies. 


Now  we  are  at  a point  in  this  meeting  where  I'm  feeling  a little 
anxious,  not  because  the  group  has  been  reduced  in  number,  because 
I expected  that  to  happen.  I also  expect  that  the  interaction  and 
the  various  influences  that  have  happened  as  a result  of  it,  have 
carried  over  here  in  this  room,  in  a form  that  is  useful.  And  I 
expected  that  a core  of  people  like  the  ones  who  are  here  now  would 
be  depended  upon  to  formulate  some  tentative  conclusions  as  to  what 
seems  to  be  needed  in  the  best  interests  of  improved  research 
utilization.  That  is  what  I hope  we  will  accomplish  this  afternoon. 
Please  take  whatever  length  of  time  is  necessary  to  formulate  some 
major  conclusions,  and  the  implications  of  those  conclusions  if  they 
were  acted  upon.  I hope  you  will  get  to  a point  where  tomorrow 
morning  we  can  make  some  clear  and  sound  recommendations  that  might 
have  impact  on  policy  formulation.  I am  hoping  that  the  kind  of 
recommendations  that  are  generated  on  the  basis  of  what  has  happened 
this  week  will  provide  the  type  of  dialogue  that  Dr.  Rostker  will  be 
yearning  to  hear  in  the  morning,  and  will  be  prepared  to  critique  and 
respond  to.  I am  not  sure  if  I am  being  clear  as  to  what  the  charge 
is  at  this  particular  time.  Ed,  why  don't  you  give  five  words  on 
what  it  is  you  think  we  ought  to  be  prepared  to  do  in  the  morning. 

DR.  GLASER:  As  we  listened  to  Dr.  Rostker  talk,  I think  a number  of 
people  in  the  research  community  felt  that  his  stance  was,  in  a sense, 
attacking  and  adversarial.  I think  there  is  an  opportunity  here  to 
have  a dialogue  with  someone  who  is  in  a major  policy  position  to 
influence  appropriations  and  other  actions  in  the  Navy.  1 think 
you  will  find  a receptive  attitude  tomorrow  morning.  A constructive 
question  to  consider  is  what  can  we  do  together  to  get  the  people 
at  this  conference  and  Dr.  Rostker  to  understand  each  other  better 
and  get  on  the  same  track.  You  have  people  here,  such  as  Bob  Smith, 
Glenn  Bryan,  and  others,  who  are  in  a position  to  exert  major  in- 
fluence in  the  situation.  I think  if  we  can  focus  on  what  can  be 
done  constructively  to  facilitate  the  bridging  between  good  RED  and 
what  Congress,  the  public  and  the  operating  forces  would  like  to 
see  as  payoff  from  this  investment,  we'll  have  made  major  progress. 

At  the  same  time,  we  perhaps  need  to  educate  all  concerned  that  in 
R&D  there  is  a certain  amount  of  "drilling."  By  this  I mean  that 
you  hope  to  hit  oil  and  may  in  fact  do  so  much  of  the  time,  but  no 
matter  how  good  a geologist  you  are,  there  inevitably  will  be,  and 
are,  some  dry  holes.  That  is  part  of  the  inherent  risk  in  RsD. 

It  is  not  a sure  thing. 

There  are  lessons  to  be  learned  even  from  efforts  that  do  not  work 
out  well;  e.g.,  we  at  least  learn  not  to  drill  in  the  same  place 
under  the  same  conditions  next  time.  So,  in  my  mind,  the  five  key 
words  in  Dr.  Sands'  charge  are:  COME  UP  WITH  HELPFUL  RECOMMENDATIONS 
--to  facilitate  the  utilization  of  that  which  is  worth  the  pro- 
motional effort.  That  implies  that  you  don't  go  out  and  try  to  market 
or  waste  your  energy  and  resources  in  pushing  for  the  utilization  of 
every  RED  output.  A certain  number  of  things  that  you  try  will  not 
turn  out  to  be  useful.  But  a kind  of  policy  question  that  you 
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conceivably  might  raise  is  how  long  do  you  drill  or  how  many  feet  j 

do  you  go  down  before  you  say,  "Well,  this  doesn't  look  very  pro-  j 

mising."  A pilot  study  may  be  a good  way  to  go  before  you  invest  j 

large  amounts  of  money,  if  it  is  possible  to  get  some  kind  of  j 

indication  of  potential  payoff  through  a pilot  effort.  You  then 
can  refine  your  approach  and  proceed  if  it  looks  worthwhile. 

Finally,  a legitimate  and  nondefensive  point  that  should  be  made 
to  GAO  and  Dr.  Rostker  is  that  knowledge,  as  Dr.  Donald  Pelz  at 

the  University  of  Michigan  has  observed,  can  be  used  in  more  diverse 

ways  than  the  term  "utilization"  ordinarily  implies.  It  can  be 
used  to  help  formulate  policy,  to  solve  problems,  or  to  implement 
programs.  Modes  of  use  can  vary  from  "instrumental,"  such  as 
assisting  a specific  action  or  decision,  to  "conceptual,"  such  as 
affecting  a decision  maker's  understanding  or  thinking  about  an 
issue.  Thus,  some  of  the  subtler  ways  in  which  R&D  findings  are 
utilized  may  not  be  easy  for  GAO  or  anyone  else  to  trace  neatly 
and  definitively. 

POLICY  RECOMMENDATIONS 

DR.  GLASER:  Well,  I think  we  might  say  welcome  to  the  survivors 
of  this  3 1/2  day  meeting.  We  appreciate  your  staying  aboard.  Let 
us  now  turn  directly  to  hear  from  each  of  the  three  Service  groups. 

I wonder  if  Colonel  Hoggatt,  the  representative  for  the  Air  Force 
group,  could  come  up  first  and  let  us  know  what  you  folks  would 
recommend . 


AIR  FORCE 


COL  R.  S.  Hoggatt,  USAF 
Commanding  Officer,  Applications  Office 
Air  Force  Human  Resources  Laboratory 
Brooks  Air  Force  Base 


What  I intend  to  say  does  not  constitute  a step-by-step  process,  but 
I think  it  is  a distillation  of  some  of  our  topics  and  of  our 
thinking.  Our  views  can  be  organized  into  seven  topical  areas: 

1.  The  first  area  has  to  do  with  the  matter  of  communication 
between  the  researcher  and  operator.  Regardless  of  whether 
the  problem  or  requirement  is  research-generated  or  operator- 
generated, there  should  be  an  iterative  process  between  the 
two  until  you  get  the  problem  so  clearly  identified  and  defined 
that  you  can  write  a "request  for  personnel  research"  (or  RPR 
as  the  Air  Force  calls  it),  which  then  gets  it  into  the  system. 
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ii. 


2.  The  RPR  itself  is  really  an  estimate  of  what  you  intend  to  do, 
and  includes  such  things  as:  the  budget  expectations;  that  is, 
how  much  money  you  expect  to  use  in  the  research;  the  objectives 
of  the  research;  the  intended  uses  and  concepts  for  use  of  the 
research;  the  major  prospective  users  and  how  it  is  going  to 

be  used.  A situational  description  on  how  the  problem  came 
up  would  be  helpful,  plus  a cost  estimate  or  indication  that 
a cost  analysis  should  or  should  not  be  made. 

3.  The  research  plan  should  include  the  detailed  plans  and  pro- 
gramming of  the  research  and  should  be  done  by  the  research 
community  itself.  It  should  contain  a delineation  of  the  logis- 
tical backup  necessary  to  support  it,  including  the  long-range 
costs  three  years  down  the  pike.  It  should  serve  as  a contract 
or  charter  between  the  research  community  and  the  operational 
community.  The  research  plan  should  also  indicate  the  technical 
forecast  risks  that  may  be  involved,  whether  or  not  results 
useful  to  the  operational  community  are  expected,  and  some  idea 
about  where  there  may  be  slips  in  this  whole  process.  These 
steps  are  pretty  well  systematized  within  the  Air  Force 
Regulation  AF-80-51. 

4.  We  felt  that  one  of  the  things  that  came  out  of  this  symposium 
was  the  necessity  for  an  ad  hoc  committee  to  oversee  the  research. 
It  might  include  the  researcher  manager,  an  applications  person, 

a facilitator,  a user  and  a technologist.  Such  a committee 
would  serve  a liaison  role  with  the  other  commands.  In  the 
Air  Force  we  are  really  pretty  separated,  but  the  Air  Training 
Command  model  would  be  a good  one  to  follow. 

5.  The  research  activity  per  se  includes  the  collecting  of  data, 
with  monitoring  by  the  ad  hoc  committee.  The  sponsors  of  the 
research  should  be  continually  advised  or  updated  each  time  a 
significant  event  occurs.  In  the  Systems  Command  we  have  a 
document  called  the  "Mazes."  Each  time  a significant  event 
occurs  within  a research  it  is  documented  there.  We  intend  to 
formalize  this  in  a forthcoming  update  of  80-51.  This  will  be 
given  to  the  research  requester  on  a periodic  basis  every  three 
months.  During  this  time  one  of  the  steps  you  may  have  to  take 
is  a possible  redirection  of  the  research  because  some  things 
you  found  out  in  the  data  collection  may  suggest  a need  for 
refinements  or  changes.  You  are  concerned  during  this  time 
with  quality  control  and  we  felt  possibly  a coalition  of  the 
researcher  and  the  operator  would  be  a good  idea.  However, 

if  you  have  enough  communications  and  you  understand  each 
other's  problems,  this  isn't  absolutely  necessary,  but  it 
was  mentioned. 

6.  In  the  research  results  step  you  analyze  and  evaluate  the 
data,  formulate  recommendations,  generate  a report  and  complete 
your  cost-benefit  analysis. 
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7.  The  Air  Force  system  needs  to  be  upgraded  in  the  applications 
area.  We  aren't  particularly  strong  here.  You  cannot  do 
it  on  a part-time  basis.  In  the  applications  process  there 
needs  to  be  some  high-level  endorsement  of  your  findings  and 
recommendations.  There  aiso  should  be  involvement  in  your 
research  projects.  There  needs  to  be  a plan  for  briefings,  and 
for  marketing  the  research.  In  this  phase  you  identify  spinoffs 
and  additional  products.  You  get  in  touch  with  transfer  agents. 
You  have  the  user  evaluate  the  product,  and  you  evaluate  it 
too,  so  it  doesn't  just  get  pushed  off  into  a corner  someplace 
and  completely  disappear  after  you  have  spent  the  time  and 
money  at  the  request  of  somebody  to  do  this.  You  then  work 
out  some  way  of  transferring  the  funding  from  the  research 
area  to  the  OSM  area.  This  can  be  shown  as  a six-section  pie, 
with  an  applications  piece  in  the  middle.  That  offers  a good 
learning  tool  for  what  we  think  should  be  done. 


DISCUSSION 

DR.  ROCKWAY:  The  major  points  we  are  trying  to  make  here  are  that 
we  think  a research  utilization  system  should  be  formal  and  it 
should  be  documented.  Particularly  in  a military  environment  you 
need  written  agreements  to  preserve  some  continuity,  because  the 
people  that  you  are  dealing  with  frequently  are  very  mobile. 

Formal  documer^tat ion  tends  to  preserve  the  continuity.  We  don't 
feel  that  the  steps  that  we  have  identified  here  are  rigid.  In 
fact,  there  has  to  be  an  awful  lot  of  flexibility  in  the  research 
process.  Our  concept  is  that  at  different  stages  throughout  the 
project  the  research  needs  to  be  supported  by  different  levels,  so 
that  we  can  bounce  around  from  the  guy  who  actually  may  be  imple- 
menting it  and  explore  other  areas  for  potential  use  as  well. 

ADDITIONAL  COMMENT  FROM  THE  AUDIENCE:  One  of  your  recommendations 
to  facilitate  utilization  is  that  you  do  a joint  evaluation  of  the 
R6D  effort  between  the  researcher  and  the  user.  This  fits  com- 
pletely with  our  experience,  as  well  as  with  the  RU  literature 
concerning  procedures  that  make  for  successful  implementation. 


DEPARTMENT  OF  LABOR  PERSPECTIVE 

MR.  JUDAH  DROB:  I hope  you  understand  the  humble  spirit  in  which  1 
read  this  proposal  that  1 wrote -- 'We  urge  DoD  to  designate  some 
person  at  a high  ievel  to  be  responsible  exclusively  for:  (1) 
stimulating  action  by  each  of  the  services  to  maximize  use  in 
planning  and  in  operation  of  relevant  RDT6B  and  of  knowledge 
derived  from  other  sources,  including  operating  data,  best  practice 
and  RDTSE  performed  by  others,  (2)  maximizing  interchange  among  the 
services  of  utilization  plans  and  experience,  and  (3)  advocating  the 
building  of  institutional  means  for  systematic  knowledge  transfer 
and  uti 1 ization. 
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We  urge  this  appointment  as  a first  step  to  signify  to  all  concerned, 
including  the  committees  of  Congress,  that  DoD  means  business  in 
promoting  utilization  of  RDT6E.  Once  such  an  appointment  is  made, 
we  urge  the  designated  person  to  consider  the  following  program  of 
act i on : 

1.  Obtain  appointment  by  each  of  the  services  of  a high-level 
counterpart  person,  the  three  to  work  closely  with  the  DoD 
designee  in  planning  further  action. 

2.  Convene  one  or  more  conferences  of  representative  RDTSE  pro- 
ducers, managers,  users  and  top  policy  makers  to  plan  for  and 
promote  utilization. 

3.  Urge  assignment  of  specialists  in  utilization  to  this  role  in 

each  RTD&E  lab,  and  in  the  appropriate  place  in  user  organizations. 

k.  Organize  linking  units  for  the  purpose  of  assisting  public 

officials  and  civilian  jurisdictions  to  seek  out  and  use  know- 
ledge, techniques,  equipment,  etc.,  developed  for  service 
organizations. 

5.  Develop  practical  means  for  storage  and  retrieval  of  RDTSE 
findings  and  products,  stimulate  and  resort  to  this  data  base 
as  a first  step  in  problem  solving,  and  arrange  foi"  synthesis 
and  analysis  of  work  in  related  areas  for  easy  reference  by 
potential  users. 

6.  Advance  understanding  of  the  capabilities  and  limitations  of 
RDT&E,  stimulate  further  development  of  techniques  of  statistical, 
cost-benefit,  and  process  analysis,  and  conduct  RDT£E  on  the 
processes  of  utilization  as  they  are  developed  in  DoD  and  the 
servi ces . 


DISCUSSION 

CAPT  CLARKIN:  The  central  concept  of  what  you  are  proposing  bothers 
me  because  it  seems  very  much  like  the  ad  hoc- ism  that  we  observed  in 
DoD  in  general.  My  contention  is  that  the  responsibility  for  imple- 
menting appropriate  and  relevant  RDT&E  already  exists  within  the 
managers  who  are  occupying  positions  throughout  the  organization.  My 
fear  is  that  if  you  do  establish  the  ad  hoc  utilization  position  that 
you  are  addressing,  these  managers  would  be  encapsulated  and  isolated 
in  the  same  fashion  that  you  find  the  management  and  performance  of 
R6D  as  it  is  carried  out  at  present.  The  analogy  would  be  the 
establishment  of  the  human  goals  concept  as  isolated  from  the  rest  of 
the  functional  organization.  I think  that  to  the  extent  it  has  been 
isolated,  it  has  been  ineffective. 

MR.  DROB:  That  certainly  is  a very  valid  point  of  view,  but  I think 
those  of  us  who  have  been  engaged  in  the  utilization  community  have 
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felt  that  nothing  really  happened  until  specific  people  had  the 
responsibility.  I think  the  question  of  how  they  tie  in  with  their 
organizations  is  critical.  And  certainly  they  should  not  be  seen 
as  outsiders;  they  should  be  performing  a function  that  is  relevant, 
but  again  1 don't  know  your  milieu  well  enough  to  engage  in  any  kind 
of  debate  with  you. 

CAPT  CLARKIN:  I can  assure  you  that  'r  the  chief  executives  of  the 
organizations  are  involved  and  concerned,  the  hearts  and  minds  of 
the  troops  wi 1 1 fol low. 

MR.  DROB:  Except  that  my  experience  with  higher  executives  is  that 

they  never  have  enough  time  to  do  all  the  things  that  their  hearts 

and  minds  are  involved  in  and  they  need  arms  which  will  spend  full 
time  at  expressing  their  interests.  And  that  is  why  I started  with 
the  notion  that  if  somebody  is  sufficiently  visible  and  sufficiently 
important,  then  people  in  the  ranks  will  have  to  understand  that  this 

i s a high  priority  for  the  top  managers.  Again,  I would  not  for  a 

minute  think  that  I knew  more  than  Capt . Clarkin  with  regard  to  what 
is  required  to  facilitate  utilization  in  the  military. 

DR.  UHLANER:  1 would  like,  from  the  Army  point  of  view,  to  very 
much  disagree  with  Judah  Drob's  proposal,  and  add  a word  to  Jim 
Clarkin's  point  by  saying  that  I think  the  proposal  is  based  a good 
deal  on  ignorance  of  how  the  Defense  Department  relates  to  the  ser- 
vices, and  what  their  authority  is.  The  point  is,  if  there  were  a 
DoD  which  brought  together  all  the  services  in  one  operating  entity, 
with  authority  to  run  its  own  DoD  personnel  system,  that  suggestion 
might  be  practicable.  But,  rather,  what  you  do  have  is  a policy- 
making body  at  DoD  levels  which  really  gives  only  guidance  in 
generalities  to  the  separate  services.  Therefore,  the  user,  in 
terms  of  final  authority  in  most  instances,  except  for  the  stimulation 
of  policy  by  the  DoD  level,  resides  at  a lower  level,  namely,  the 
services.  Your  proposal,  Judah,  is  aimed  at  strengthening  the  RU 
function  at  the  point  where  authority  for  the  actual  operation  of 
personnel  and  training  resides.  That  kind  of  individual  would  be  in 
the  respective  services,  not  in  DoD.  DoD  has  no  authority  in  terms 
of  the  operation,  other  than  guidance. 

Thus,  I would  strongly  echo  Jim's  point.  With  ignorance  it  looks  like 
a great  thing  to  go  as  high  up  as  you  can,  except  that  when  you  go 
up  too  high  and  there  is  really  no  functional  authority,  you  are 
essentially  joining  a policy-making  group  that  deals  only  in 
genera  1 i t ies . 

MR.  DROB:  Is  this  kind  of  approach  applicable  at  the  service  level? 
Might  that  be  a possibility?  What  I was  concerned  with  initially 
was  making  visible  the  concern  that  Dr.  Rostker  expressed  for  an 
increased  awareness  and  activity  in  the  utilization  area,  and  it 
seemed  to  me  that  this  was  one  to  do  it. 


-203- 


ARMY 


DR.  DRUCKER;  Army  R6D  has  been  wrestling  for  over  10  years  with  the 
problem  of  how  to  improve  research  utilization.  In  this  discussion 
we  will  be  talking  primarily  about  6.3  research. 

From  time  to  time  in  this  development  over  the  last  10  years,  several 
conditions  have  occurred  to  us  that  would  have  to  be  satisfied  and 
many  of  them  we  have  touched  upon  this  week.  Let's  call  them 
assumptions.  For  example,  we  recognize  a need  for  an  orderly 
system  to  accumulate  and  prioritize  research  requirements.  Par- 
ticularly, we  are  anxious  that  such  prioritization  not  be  done  by 
the  researcher--a  role  that  seems  to  have  fallen  to  us  all  too 
frequently  over  the  past  10  years--a  role  we  did  not  particularly  want. 
We  wanted  to  be  executors  of  the  program  and  not  policy  makers.  We 
wanted  programs  consistent  with  budgetary  considerations,  budget 
allocations,  recognition  of  the  6. 1.2. 3 constraints,  and  the  review 
procedures  of  budgets  and  the  budget  cycles.  We  wanted  to  involve 
the  consumer  earlier,  and  hopefully  we  wanted  to  involve  all  the 
potential  consumers  who  would  be  ultimately  involved  in  the  utilization 
process.  And  very  importantly,  we  wanted  our  total  programs  to 
be  consistent  with  guidance  documents--for  example,  guidance  documents 
from  DoD,  from  the  Army,  and  our  own  five-year  plan--as  well  as 
with  the  consumer  needs.  But  we  wanted  the  program  primarily  to 
respond  to  these  guidance  documents,  and  also  to  specific  research 
requirements.  First  the  guidance,  then  the  requirements.  We 
wanted  to  establish  that  our  programs  are  not  driven  exclusively 
or  primarily  by  the  user  requirement,  which  did  happen  over  a fairly 
long  span  of  years.  In  general,  we  wanted  to  establish  formally 
that  Army  consumers  and  laboratories  have  some  common  understanding 
of  the  nature  of  the  research  process,  what  products  are  to  be 
expected  from  research  effort,  and  in  what  time  frame. 

The  fact  that  we  have  changed  our  procedures  from  year  to  year  would 
give  testimony  to  the  fact  of  dissatisfaction  with  this  arrangement. 
Here  are  the  things  that  i think  are  worth  mentioning  from  AR-8 
entitled  "Personnel  Performance  and  Training  Program." 

We  have  in  it  a topic  of  generation  of  RDTSE  needs  and  objectives. 

The  set  established  for  the  requirements  people  in  this  AR  is  given 
in  terms  of  the  ARI  five-year  plan  for  advanced  development.  Our 
boss,  the  DCSPER,  asks  for  these  requirements  in  terms  of  human 
resource  needs,  but  consistent  with  the  five-year  plan  and  other 
guidance  documents.  The  responsibility  for  the  annual  review  and 
approval  is  vested  not  with  the  research  community,  but  with  the 
DCSPER  himself,  our  military  boss,  for  both  the  6.1  and  6.2  programs 
as  one  entity,  and  for  the  6.3  as  a second.  His  sign-off  is  on  the 
formal  documents. 

Another  important  paragraph  deals  with  preparing  the  advanced 
development  plan  and  executing  the  program.  V/e  talk  about  such 
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things  as  identification  of  the  DoD  project  line  reference  to  the 
particular  HRN  or  the  requirement.  We  talk  about  meetings  between 
the  sponsor  representative  and  the  principal  investigator  from  the 
developing  agency.  Mentioned  there  specifically  is  the  requirement 
for  arranging  for  military  support  by  the  sponsor,  and  a request 
for  semiannual  status  advice  by  the  sponsor  if  he  so  desires.  We 
have  an  appendix  which  tells  him  exactly  how  to  do  that.  The 
sponsor  responsibility  for  implementation  is  also  spelled  out. 

This  is  specifically  with  direct  involvement  of  the  developing 
agency.  Finally,  in  chapter  3 there  is  mention  of  an  in-process 
review  which  can  be  at  the  request  of  the  sponsor.  This  is  a formal 
review  reserved  for  those  instances  where  you  have  some  issue  that 
needs  resolution  and  neither  the  sponsor  nor  the  developing  agency 
can  handle  it.  It  is  then  brought  before  the  DCSPER,  who  attempts 
a resolution.  The  regulation  does  encourage  a series  of  informal 
interactions  between  the  investigators,  the  developing  agencies, 
and  the  sponsor  representatives.  That's  at  the  working  level  and 
with  no  particular  formal  documentation. 

To  ensure  direct  application  of  RDTSE  findings  to  the  military 
mission,  a paragraph  is  given  over  to  interpretation  and  assistance 
in  utilization  of  findings.  This  is  entitled  "Relationship  of  the 
Implementing  Agency  and  the  Developing  Agency  in  Utilization 
Activities."  The  implementing  agency  may  request  from  the  developing 
agency  additional  assistance  concerned  with  such  things  as  detailed 
planning,  contract  planning,  selecting  a contractor,  or  even  writing 
the  proposal  for  the  contract. 

I mention  these  items  because  they  are  specifically  in  the  AR.  It 
is  down  to  that  kind  of  detail.  Then  there  is  a section  on  addi- 
tional findings  determined  to  have  Army-wide  application.  The 
developer  supplements  the  RDT&E  report  with  such  things  as  methods 
by  which  the  findings  were  incorporated  in  the  military  system  and 
recommendations  for  additional  implementation;  methods  to  adapt 
findings  further  into  the  implementing  documents;  measures  recommended 
to  gain  acceptance  and  promote  actual  practice  of  findings  within  the 
community  and  recommendations  for  additional  RDT&E.  We  are  en- 
couraged to  disseminate  RDTSE  reports  widely.  In  addition,  ARI 
has  instituted  what  it  calls  the  RDT&E  utilization  report,  which 
is  a synopsis  containing  useful  findings  from  contract  reports  and 
how  the  product  or  finding  was  utilized  by  the  operational  community 
and/or  the  research  community.  The  sponsor  has  the  obligation 
to  report  on  his  utilization. 


DISCUSSION 

QUESTION;  Is  your  sponsor  always  your  user? 
DR.  DRUCKER:  No,  not  necessarily. 
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QUESTION:  Why  should  the  sponsor  report  instead  of  the  user? 


DR.  DRUCKER:  I have  been  using  the  terms  interchangeably  here. 

Actually,  we  talk  more  in  terms  of  the  sponsor  representative  during 
the  stage  of  research  execution,  and  the  user  during  utilization.  When 
results  of  a sponsored  RDT6E  or  significant  phase  of  a DA  or  DoD  project 
are  available,  the  sponsor,  after  a suitable  interval  of  time  for 
consideraton  or  trial,  will  complete  and  submit  to  ARI  a report  of 
utilization  using  a suggested  format.  An  appendix  talks  about  the 
product;  paragraph  ^ of  that  appendix  actually  gives  a litany  of 
the  types  of  products  that  might  be  involved  in  the  report.  For 
example,  specific  products  might  be  changes  to  a program  of  instruction 
or  changes  to  a policy  or  to  an  Army  regulation;  or  a new  or  revised 
test  or  training  circular  or  DA  pamphlet;  or  a revised  operational 
instruction  or  decision  or  doctrine;  or  input  to  management  decisions. 

The  GAO  in  their  recent  report  cited  these  as  products  or  uses.  The 
scope  or  impact  of  change  implied  by  the  product  is  also  included  on 
the  form  that  the  sponsor  must  fill  out.  Does  the  product  have  Army- 
wide implications,  for  instance?  Does  it  affect  all  of  infantry? 

What  sort  of  improvement  in  operations  does  it  entail?  The  example 
given  effects  ^0  percent  improvement  in  the  effectiveness  of  combat 
arms  during  night  operation.  And  finally,  very  significantly,  if  the 
product  is  not  used  the  sponsor  must  say  why  not,  or  whether  it  is 
possible  that  i t wi 1 1 be  used  at  some  future  date.  We  think  that 
this  program  will  satisfy  a majority  of  the  GAO  research  management 
recommendations.  They  are  not  completely  satisfied,  however.  For 
example,  they  say  that  what  we  urgently  need  are  criteria  to  identify 
results  intended  for  implementation  as  opposed  to  those  intended  to 
serve  the  needs  of  knowledge  generating  or  technological  base  efforts. 

We  are  working  on  this.  I will  conclude  with  that  and  ask  Jay  Uhlaner 
if  he  would  like  to  add  anything  to  what  I have  said. 

DR.  UHLANER:  The  sponsor  is  usually  at  a higher  level  of  headquarters, 
so  the  TRADOC  might  be  sponsor,  but  the  users  might  be  all  the  staff 
elements  of  TRADOC,  or,  for  that  matter,  users  might  be  elements 
of  other  commands.  But  the  responsibility  for  all  of  the  formal 
decision  aspects  would  be  with  the  headquarters  sponsor;  it  would 
be  almost  impossible  to  go  out  and  talk  to  all  the  different  sub- 
elements and  get  some  kind  of  agreement  and  consensus  action  otherwise. 
So  the  headquarters  sponsor  has  that  responsibility. 

Years  ago  in  this  research  we  very  frequently  had  multiple  sponsors 
and  we  would  list  them,  and  we  found  something  very  curious,  which 
I am  sure  everybody  else  who  has  worked  in  this  has  experienced. 

What  you  found  was  the  researcher  was  right  in  the  middle  between 
these  multiple  sponsors;  we  evolved  a system  of  a principal  sponsor 
and  then  finally  a single,  high-level  sponsor  for  given  programs. 

We  found  that  worked  best;  this  principal  sponsor  has  to  argue 
it  out  with  all  the  other  users  with  or  without  us  present.  That 
seems  to  work  a lot  better. 

QUESTION:  Has  this  arrangement  facilitated  utilization? 
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DR.  UHLANER:  Very  much  so.  The  changes  that  are  in  this  AR,  in 
many  respects  are  relatively  minor.  There  is  one  major  one  from 
our  point  of  view,  but  not  from  the  Army's  point  of  view,  in  that  we 
have  more  documents  to  feed  the  five-year  plan.  But  I think  that 
the  rest  of  the  Army  is  now  not  particularly  interested  in  that 
aspect.  Essentially,  in  the  10  years  that  it  has  been  operating, 

I would  say  it  has  definitely  facilitated  the  utilization  of  the 
product.  I think  it  did  more  than  that.  It  also  made  it  possible 
to  have  a more  balanced  program.  What  isn't  written  into  the 
regulation  is  that  we  normally  in  our  review  cycles  don't  speak  to 
one  set  of  sponsors  at  a time.  Instead,  we  have  the  DCSPER  convene 
all  the  sponsors.  What  happens  is  that  they  have  three  days  and 
nights  to  sort  it  out.  They  then  come  to  some  kind  of  resolution 
in  terms  of  relative  priorities.  Then  you  get  down  to  business. 

There  is  a principle  here  that  you  should  not  be  dealing  individually 
with  one  sponsor  at  a time  as  far  as  the  programmatic  part  of  it 
is  concerned,  because  they  all  have  many  more  requirements  than 
you  possibly  have  resources.  At  least  that  is  our  experience. 

QUESTION:  Not  all  your  research  projects  are  documented  through 
this  route,  is  that  correct?  All  of  them  have  a sponsor  who  has  a 
responsibility  to  work  with  ARI,  correct? 

OR.  DRUCKER:  Are  you  referring  to  6.3? 

RESPONSE:  Yes,  we  sort  of  face  the  same  concern.  It  seems  that 
some  parts  of  the  R&D  have  to  be  technology  based  and  thus  expect 
to  be  put  into  this  tight  a mold. 

Another  basic  question,  let's  say  the  GAO  did  the  same  exercise 
five  years  from  now  and  they  used  the  same  procedure,  looking  at 
technical  reports  and  most  of  your  technical  reports  were  done  under 
6.1  and  6.2.  It  wouldn't  look  much  different,  would  it? 

DR.  DRUCKER:  1 believe  what  they  just  did  was  an  unhappy  sampling 
when  they  found  only  28  of  our  products  and  our  reports  reflecting 
6.3,  subject  to  these  particular  procedures.  In  actuality,  I 
think  that  they  would  find  a higher  percent  of  this  kind  of  product. 

I think  my  answer  to  you  is  that  if  we  did  nothing,  the  answer 
would  be  yes,  they  would  find  the  same  problems.  Something  Joes 
have  to  be  done. 

COMMENT:  My  point  is  that  working  for  the  Department  of  Defense, 
we  can  define  a whole  spectrum  of  users. 

DR.  DRUCKER:  I briefly  mentioned  the  responses  to  guidance  documents 
which  have  to  do  with  the  future  of  the  establishment,  that  is  one 


point.  The  other  is  that  within  the  military  establishment,  the  Chief 
of  Staff  or  the  CNO  is  as  much  responsible  for  the  research  as  he 
is  for  the  troops.  This  is  quite  unique  in  comparison  with  other 
federal  agencies,  where  the  U.  S.  Office  of  Education  or  NIE, 
for  example,  doesn't  have  any  troops.  So  the  change  process  is  a 
little  bit  different  where  NIE  is  trying  to  influence  the  local 
school  system.  Within  the  military  establishment,  we  all  work  for 
the  same  people.  I think  we  can  learn  about  the  change  agent,  but 
the  way  he  operates  within  his  system  is  a little  different.  So 
trying  to  tie  these  points  together,  in  my  thinking  we  in  the  military 
have  a chance  to  operate  within  a long-term  time  spectrum  reflecting 
the  guidance  documents.  And  thus,  the  user  is  not  just  concerned 
with  today's  problem,  but  rather  the  anticipated  problems  for  the 
next  5,  10  or  15  years.  And  we  can  look  to  an  overall  concern  if 
the  Chief  of  Staff  or  the  CNO. 

QUESTION:  During  the  past  few  years  ARI  has  been  implementing  the 
policy  of  collocating  their  research  with  the  user  commands.  Has 
this  immediate  contact  between  the  researchers  and  the  users  re- 
sulted in  increased  utilization? 

DR.  DRUCKER:  I would  say  definitely,  yes.  In  all  stages  of  the 
program  cycle,  it  has  obvious  soci al -psycholog i cal  advantages 
as  well  as  benefits  to  the  utilization  of  the  products.  As  far  as 
utilization  is  concerned,  our  people  there  at  the  installations 
serve  in  a direct  role,  much  more  direct  than  would  be  possible  if 
we  operated  from  headquarters  alone. 

QUESTION:  Do  those  field  units  tend  to  be  service  organizations  for 

the  local  command,  or  do  they  still  do  research? 

DR.  DRUCKER:  In  part,  that  is  what  I call  helping  to  pay  the  rent; 
to  help  them  with  their  local  problems  as  well  as  conduct  their 
portion  of  the  overall  long-range  research  program.  But  we  are 
much  more  permissive  in  the  amount  of  time  they  are  allowed  to  spend 
at  that  sort  of  thing.  Maybe  "allowed  to  spend"  is  not  right,  but 
it  all  works  out  well. 

DR.  GLASER:  In  connection  with  a point  you  just  made,  there  is  a lot 
of  effort  here  to  provide  what  industry  would  call  technical  assistance. 
That  is,  you  spoke  about  the  developer  and  researcher  being  available 
to  the  sponsor  to  carry  through.  In  industry  you  will  often  find 
that  when  the  research  group  develops  something,  generally  one  of 
the  key  persons  who  has  been  involved  in  this  development  within 
the  research  department  now  moves  over  to  manufacturing  for  a period 
of  time  to  help  with  the  implementation  process  if  the  manufacturing 
department  wants  any  help. 

Bert  Griffis  has  some  comments  he'd  like  to  make. 


-208- 


r • ^ 


REHABILITATION  SERVICES  ADMINISTRATION  PERSPECTIVE 

BERT  GRIFFIS:  I do  have  seven  comments  carefully  distilled  from 
close  attention  to  all  that  has  transpired.  I do  appreciate  the 
invitation  to  attend  and  after  three  days  I feel  better  educated,  if 
not  saturated.  My  seven  comments  (with  tongue  in  cheek)  are; 

1.  If  it  ain't  broke,  don't  fix  it.  I can  tell  you  1 have  had 
many  occasions  in  my  life  to  regret  not  having  left  well 
enough  alone,  and  this  may  be  another  one  of  them. 

2.  Try  to  find  out  what  is  already  being  done  well,  then  carefully 
package  and  document  it  and  diffuse  it  more  widely  for  some- 
what more  general  use.  To  have  come  this  far  we  must  have 
been  doing  some  things  right.  For  this  idea,  I am  indebted 

to  Professor  Ronald  Lippit  of  the  University  of  Michigan. 

3.  Offer  handsome  rewards  for  thorough,  precise  explications  and 
airings  of  all  sticky,  embarrassing  problems.  This  will 
completely  destroy  whatever  establishment  you  have  and  enable 
you  to  start  over  in  a thoroughly  innovative  fashion. 

According  to  Ed  Glaser,  the  Vietnamese  almost  beat  us,  in 
part,  by  employing  this  principle. 

Do  not  be  afraid  to  be  a bit  subjective.  Avoid  being  over- 
objective,  lest  it  make  a fool  of  you.  Remember  the  story  told 
by  Kierkegaard  of  an  inmate  who  escaped  from  an  asylum  and 
started  walking  to  town.  On  the  way  he  began  to  wonder,  "How 
am  1 going  to  prove  that  I am  sane?  Let's  see,  I must  come 
up  with  something  indubitably  true.  I must  state  it  precisely 
and  that  will  prove  my  sanity."  So  he  picked  up  a large, 
round  stone,  turned  it  over  carefully  in  his  hand,  observed  it 
for  a while,  then  exclaimed,  "I  have  itl  The  earth  is  round." 

So  he  put  the  stone  in  his  hip  pocket,  and  resumed  his  way 
toward  the  town.  And  with  each  step  he  took,  the  stone  hit 
his  fanny,  reminding  him  that  the  earth  isn't  that  round; 
so  that  as  he  entered  the  town  he  was  repeating  over  and  over 
to  himself  with  every  step,  "The  earth  is  round,  the  earth 
is  round,  the  earth  is  round." 

5.  Beware  of  systems  and  models,  bearing  in  mind  what  all  good 
philosophers  have  said,  I mean,  that  all  systems  are  probably 
premature,  and  remembering  also  what  the  incomparable  Santayana 
said,  namely,  that  chaos  may  be  at  the  bottom  of  it  all. 

6.  Do  not  expect  to  change  the  opinion  of  any  man,  for  you  probably 
can't  do  it.  Both  Jefferson  and  Franklin  held  this  as  a first 
principle  and  refused  to  argue  with  men  of  fixed  opinion, 
knowing  it  to  be  a waste  of  time.  Instead  they  framed  our 
constitution  in  such  a way  that  all  fixed  opinions  can  express 
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themselves  as  best  they  can  and  in  the  process  do  as  little  harm 
and  as  much  good  as  possible. 


7.  Do  not  forget  for  a moment  what  the  existentialists  have  said, 
namely,  that  all  of  us  are  under  sentence  of  death  and  have 
only  reprieves  of  varying  length.  In  other  words,  recognize 
the  human  factor  on  all  occasions  and  you  will  be  amazed 
at  the  response  you  get.  What  I mean,  somewhat  in  the  fashion 
of  Thoreau,  is  this  quote,  "If  you  will  advance  confidently 
toward  your  dreams  you  will  meet  with  a success  undreamed  of 
in  common  hours." 

I 

NAVY 

DR.  SMITH:  The  Navy  Human  Resources  R6D  program  is  both  relevant 
and  cost  effective.  It  is  relevant  to  Navy  requirements  posed  by 
the  Chief  of  Naval  Operations.  Remember  that  Hank  Taylor  reported 
that  his  revision  of  the  GAO  figures  cited  a 75  percent  utilization 
rate  of  those  things  that  were  intended  for  use.  The  engineers  whom 
I work  with  in  my  office  think  that  is  an  absolutely  fantastic 
success  rate  compared  to  what  they  see  in  terms  of  utilization  in 
their  fields.  However,  I pressed  them  for  data,  and  they  don't 
have  any.  Back  in  the  office  I have  some  rather  rough  estimates  of 
the  potential  cost  savings  or  cost  avoidance  of  the  6.3  program, 
with  which  I am  most  familiar.  This  adds  up  to  well  over  1.5 
billion  dollars  for  a proposed  program  of  about  16  million  dollars 
for  FY78.  This  does  not  count  fuel  savings  as  a result  of  our 
work  in  simulation,  nor  increases  in  operational  readiness,  nor  the 
prospect  of  saving  several  lives  a year. 

Now  I see  our  problem  as  becoming  even  more  effective  in  utilization 
so  that  the  Navy  has  an  opportunity  to  reap  the  full  measure  of 
the  potential  benefits  that  are  involved  in  our  R&D  program.  In 
other  words,  our  group  was  very  fortunate  (in  our  meeting  yesterday 
to  prepare  this  report);  we  were  through  before  we  left  the  room.  So 
we  didn't  really  have  to  do  it  all  like  the  Air  Force  did.  Never- 
theless, from  that  meeting  we  got  some  excellent  ideas  for  doing 
things  better. 

1.  First,  we  feel  there  is  a need  for  early  and  continual  joint 
planning  for  utilization  between  the  user  and  researcher 
groups.  Under  this  specific  point  we  might  include  identifi- 
cation of  all  the  people  who  need  to  be  kept  informed  about 
the  research;  identification  of  all  the  end  products  required 
for  effective  utilization;  development  of  solid  data  about 
the  costs  and  benefits  of  the  utilization  of  each  product; 
and  identification  of  procurement  and  O&M  funds  and  billets 
that  might  be  needed  for  implementation,  so  that  these  can 
be  put  in  the  POM  process. 


2.  Our  second  major  suggestion  is  that  we  need  to  increase  the 
resources  devoted  to  utilization.  We  need  to  establish  guidance 
to  the  R6D  community  as  to  the  importance  of  utilization  and 

the  legitimacy  of  this  purpose.  We  need  to  assign  application 
officers  to  key  user  agencies  with  the  aim  of  assisting  in 
utilization.  These  duties  would  include:  the  review  of  R6D 
reports  to  identify  those  that  are  promising;  liaison  with 
the  R&D  community;  and  the  performance  of  quality  control  on 
both  RsD  and  implementation. 

3.  The  next  point  is  that  we  need  to  provide  for  technical  advisory 
service,  from  the  R&D  community  to  the  user  community,  to  deal 
with  short-term  problems  and  technology  transfer.  This  is 
especially  important  in  getting  the  second,  third  and  fourth 
utilization  out  of  existing  R&D.  Specifically,  it  was  suggested 
that  some  limited  amount  of  R&D  funds,  such  as  five  or  six 
percent,  be  used  for  this  purpose,  and  that  greater  use  of  the 
Navy  scientific  assistance  program  for  this  purpose  be  explored. 

k.  Our  fourth  point  is  with  regard  to  the  submission  of  operational 
requirements  from  potential  users  and  sponsors  as  called  for  in 
OPNAV  Instruction  5000. ^2A.  This  emphasizes  weapon  systems 
planning,  and  provides  adequate  procedures  for  passing  re- 
quirements to  the  R&D  community.  We  have  found  that  the  system 
is  responsive  to  the  special  needs  of  our  kind  of  R&D. 

5.  Next,  the  technical  report  should  contain  an  executive  summary 
that  clearly  identifies  the  anticipated  user  or  users  and  the 
expected  use  to  be  made  of  the  report.  It  should  also  contain 
cost-benefit  information,  and  the  body  of  the  report  should  show 
the  methodology  used  in  obtaining  that  information.  It  is 
recognized  that  the  R&D  community  may  be  a user  of  R&D, 
especially  as  a project  moves  from  research  to  exploratory 

and  advanced  development.  Here  is  a specific  example:  In  our 
human  factors  engineering  program  over  the  past  few  years  we 
have  been  doing  work  which  evaluates  and  improves  the  training 
capability  of  a device  called  the  Air  Combat  Maneuvering  Range. 

The  Air  Force  has  one  just  like  it  called  the  Air  Combat 
Maneuvering  Instrumentation.  What  I am  engaged  in  right  now 
is  making  sure  that  the  lessons  that  we  learned  from  that 
effort  are  applied  in  a new  program;  nameiy,  to  develop  a sea 
air  combat  maneuvering  range.  That  is  one  example  of  how  the 
R&D  community  can  serve  us. 

6.  The  next  point  is  that  there  should  be  an  annual  report  of 
utilization,  the  purposes  of  which  are  to  document  the  use  of 
R&D  and  to  encourage  additional  utilization.  This  report  is 
envisioned  as  containing  descriptions  of  the  R&D  that  was 

used,  descriptions  of  the  R£D  that  is  ready  for  use,  and  abstracts 
of  reports  of  possibly  wide  interest. 
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7.  Finally,  following  the  recommendations  of  the  GAO,  there  should 
be  a monitoring  system  for  tracking  utilization  from  a high 
headquarters  level.  Those  are  our  seven  points. 


DISCUSSION 

COMMENT:  The  last  recommendation  sounds  vaguely  like  the  published 
reports  called  Advanced  Technological  Concepts  and  Technological 
Possibilities.  Am  I wrong  in  that  comparison? 

DR.  SMITH:  You  may  be  referring  to  what  is  now  known  as  the  Advanced 
Systems  Concepts,  which  convey  the  basic  concepts  on  which  the 
R6D  programs  operate.  There  is  a requirements  "pull"  process,  in 
which  the  sponsor  generates  a requirement,  and  a technology  "push" 
process,  by  which  the  R&D  community  says,  "Based  on  our  development 
of  technology,  here  is  a thing  we  think  that  we  can  develop  for  you 
that  will  be  useful.  Are  you,  Mr.  Sponsor,  interested?"  What  I 
have  in  mind  is  monitoring  what  happens  to  it  after  it  is  developed, 
and  monitoring  the  way  it  acutally  gets  put  into  use. 

It  has  been  my  experience  that  the  Advanced  Systems  Concepts  are 
looked  upon  as  a very  poor  way  of  pushing  technology  prospects  into 
the  fleet.  The  success  record  is  not  that  good.  There  is  a marketing 
problem  associated  with  this.  Those  that  have  been  successful  have 
generally  been  successful  because  there  has  been  a long-term  process 
of  marketing  in  order  to  enter  the  POM  cycle  at  the  proper  time. 

1 don't  hear  a lot  of  this  business  of  marketing  coming  through 
this  meeting,  and  I don't  think  it  can  be  over-emphasized. 

DR.  CRAWFORD:  Having  spent  some  time  in  the  marketing  process,  I can 
assure  you  that  some  of  the  concepts  of  marketing  are  embodied  in 
these  proposals,  but  not  in  those  terms. 

One  of  the  points  under  the  development  of  utilization  plans  was 
the  identification  of  all  those  people  wno  need  to  be  kept  informed 
as  to  the  progress  of  work,  and  this  invariably  means  several 
varieties  of  people  at  different  levels  in  an  organization.  There 
is  the  individual  at  the  higher  headquarters  who  will  need  to  give 
concept  approval;  there  are  other  kinds  of  staff  people  who  will 
need  to  work  out  the  administrative  details;  and  down  below  in  the 
training  world,  there  are  some  worki ng- 1 evel  instructors  who  need 
to  be  trained  to  make  the  thing  work  or  else  it  will  never  fly. 

I think  another  point  on  your  operational  requirements  generation 
should  be  brought  out;  namely,  it  is  perfectly  possible  for  a 
researcher  to  develop  an  operational  requirement.  It  must  be  sent 
for  approval  by  the  proper  official,  however,  in  order  to  become 
an  operational  requirement  in  reality. 

OR.  SMITH:  This  is  correct. 
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QUESTION:  So  if  he  sees  a use  for  some  bit  of  research  he  can 
write  an  operational  requirement  and  get  it  approved. 

DR.  SMITH:  That's  right. 

DR.  SANDS:  Dr.  Rostker  graciously  has  accepted  our  invitation  for 
him  to  return  before  closing  to  express  his  reactions  to  what  he 
has  heard  here. 

DR.  ROSTKER:  I'm  afraid,  at  least  from  where  I sit,  this  still  sounds 
like  the  first  day  of  the  meeting.  This  last  session,  I understood, 
was  to  present  policy  recommendations.  What  I have  heard  from  the 
Army,  and  certainly  from  the  Navy,  was  a dissertation  on  present 
procedures  and  why  they  are  working,  and  that  is  where  I came  in. 

I know  that  in  the  Navy,  for  instance,  there  is  a fair  amount  of 
dissatisfaction  with  the  present  procedures. 

In  the  Navy  Secretariat  we  are  in  the  process  of  staffing  an  in- 
struction 5000,  which  is  supposed  to  provide  further  procedures  for 
giving  oversight  to  the  human  resources  and  medical  R6D  areas.  It's 
now  a hodge-podge  of  lack  of  control,  of  lack  of  single  purpose 
direction.  I think  that  is  one  of  the  problems  that  we  have  in  the 
Navy  that  we  must  solve.  So  I am  dissatisfied  when  1 hear  that  the 
present  procedures  are  just  right  — though  I really  can't  speak  for 
the  Army  or  the  Ai r Force. 

I think  there  is  a lack  of  candidness  in  the  areas  that  I am  familiar 
with  as  to  how  we  are  doing.  I would  have  hoped  that  at  the  end  of 
the  sessions  here  we  would  have  had  more  positive  policy  recommendations 
as  to  where  we  should  be  going  and  what  we  should  be  doing,  rather 
than  Just  a reading  off  of  present  procedures.  So  I for  one  am  not 
very  optimistic. 

DR.  MALOY:  I,  too,  thought  there  has  been  too  much  preoccupation  with 
policies  as  they  are  today,  in  contrast  to  where  we  might  be  going. 

I suppose  my  own  feeling  is  this:  I don't  speak  for  all  the  operators, 
but  I do  take  my  share  of  responsibility  for  the  fact  that  the  operators 
haven't  done  their  part  in  this,  and  I think  that  we  have  been 
reluctant.  1 do  believe  there  is  no  single  way  or  single  procedure 
for  improving  utilization,  rather  there  probably  are  a whole  bunch 
of  different  ways.  We  ought  not  to  be  satisfied  with  directives 
alone,  thinking  that  they  will  bring  about  what  we  want  brought  about 
as  such;  they  won't.  What  we  ought  to  be  looking  for  are  still  more 
imaginative  kinds  of  public  forums  where  policies  can  be  raised  and 
debated  as  issues  on  the  floor. 

DR.  UHLANER:  I would  like  to  comment  that  we  are  not  dealing  with 
a new  problem,  but  one  that  is  quite  old.  We  are  dealing  with  a 
problem  where  different  approaches  have  already  been  experimented 
with  and,  I would  have  to  add,  at  least  in  the  case  of  the  Army, 
that  many  of  those  have  failed.  To  be  sure,  over  the  years  some  of 
the  approaches,  such  as  the  establishment  of  field  units,  starting 
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back  from  a time  when  FCRCs  got  established,  have  seemed  to  be  a 
move  in  the  right  direction.  Much  of  what  we  are  talking  about, 
however,  is  really  not  a simple  procedural  matter.  The  problem  is 
not  very  different  from  what  you  have  in  a contract  of  marriage. 

You  work  at  it  for  a lifetime.  You  either  make  it  work  or  it 
doesn't.  I see  much  of  the  user-manager-researcher  relationship  as 
being  of  that  nature.  I think  any  one  of  those  procedures  are  very 
cold,  and  it  may  take  years  of  trial  and  back-and-forth  refinement 
effort  to  translate  procedure  into  an  operational  modus  operand!  for 
achieving  what  we're  seeking.  I also  think  that  if  we  attack  only 
utilization,  it  will  not  be  very  meaningful.  You  cannot  take  it 
out  of  context.  Sometimes  you  have  the  most  beautiful  utilization 
of  a product,  but  it  may  be  accomplished  at  the  expense  of  good 
research. 

The  message  I would  like  to  leave  is  that  I think  we  are  in  an 
experimentation  mode  and  I am  not  convinced  that  what  might  work 
effectively  under  one  set  of  conditions  in  one  place  might  necessarily 
be  the  optimum  in  another  place.  So  in  the  final  analysis  we  are 
really  very  much  dependent  on  the  skill,  the  talent,  the  capability, 
the  dedication,  the  good  faith>  and  the  integrity  of  the  people  who  are 
links  in  this  system  meaning  at  all  levels — the  users  and  the 
operators.  And  I think  you  will  find  more  of  this  kind  of  dedicated, 
able  person  at  all  levels  of  the  system-- i ncludi ng  the  users,  the 
managers,  and  the  researchers  — than  you  normally  find  in  almost  any 
other  part  of  our  American  Society.  They  all  seem  to  stay  with  their 
responsibilities,  and  are  dedicated  to  meeting  the  objectives  that 
I have  mentioned  earlier.  When  you  have  that  kind  of  good  start, 
then  I think  you  can  work  around  the  procedures.  You  are  right. 

Dr.  Rostker,  I think  procedure  itself  is  merely  a starting  point,  and 
within  a procedure  there  would  almost  be  a case  history,  and  one  would 
have  to  go  into  it  and  say  that  this  one  worked  in  this  decade,  but 
didn't  in  that.  I for  one  would  have  to  say  that  we  probably  are 
not  really  dealing  with  a science  but  more  with  an  art. 

COMMENT:  As  one  of  the  local,  unsuccessful  members  of  the  Navy  working 
group,  I would  like  to  say  that  the  part  in  our  report  that  was  a 
little  disappointing  is  that  I think  we  did  not  address  what  was 
just  referred  to  in  the  previous  discussion  as  analogous  to  marriage. 

I think  one  of  the  things  that  makes  a marriage  successful  is  that 
the  parties  are  clearly  identifiable  by  name  in  the  contract.  You 
don't  have  the  kind  of  accountability  in  the  R&D  process  that  tracks 
the  initiator  of  the  request  for  research,  and  the  development  agent, 
through  the  continuation  of  that  process,  and  holds  people  by  name 
or  by  agency  accountable  for  implementation. 

It  seems  to  me  that  one  of  the  gains  of  this  conference  is  putting 
the  problem  of  achieving  greater  utilization  of  validated  knowledge 
in  a much  sharper  focus  than  I think  has  been  true  in  the  past.  If 
that  is  so  and  there  is  a sincere  dedication  to  achieving  Letter 
means  for  moving  toward  that  objective,  that  gives  us  some  ground 
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for  some  realistic  hope  that  considerable  improvement  will  be  made-- 
not  just  in  a three-day  conference,  but  as  you  keep  working  at  the 
problem. 


COMMENT:  Most  of  the  policy  recommendations  I've  heard  center 
around  management  processes,  probably  because  9 out  of  10  people  in 
this  room  are  managers.  My  own  experience  as  a researcher  has  been 
that  if  research  gets  used,  it  is  not  because  of  some  action  taken 
by  managers.  I can't  think  of  a single  instance  in  my  own  career 
in  which  that  happened--where  some  manager  took  action  that  resulted 
in  the  operational  use  of  a product  that  I or  my  company  produced. 

I can,  however,  think  of  several  instances  where  management  either 
prevented  or  made  it  very  difficult  to  move  to  the  utilization  phase. 
The  use  of  research,  if  it  happens  at  all,  normally  happens  because 
of  the  initiative  taken  by  the  researchers  and  the  immediate  consumers 
of  that  research,  and  by  their  direct  dedication  to  the  mutually 
recognized  problem.  It  is  not  going  to  happen  through  reporting 
procedures.  It  is  not  going  to  happen  through  policy-making  bodies 
or  committees.  It  is  only  going  to  happen  if  you  can  get  the  line 
troops--the  people  who  are  carrying  out  the  research  and  the  immediate 
consumers  of  that  research--together , with  several  things  happening. 

One  thing  that  has  to  be  present  is  a clear  recognition  of  what  the 
operational  problem  is.  I do  not  mean  the  general  global  problem 
of  increasing  operational  effectiveness.  It  has  to  be  a good  deal 

more  specific  than  that.  It  has  to  be  a problem  that  is  recognized 

by  the  operator  as  being  a high-priority  problem.  It  cannot  be  a 
trivial  problem,  even  though  it  may  be  of  interest  to  the  researcher. 

Second,  the  problem  has  to  have  a feasible  RSD  solution.  There  are 
a great  many  operational  problems  that  are  of  immediate  concern  to 
the  operational  community  in  which  they  call  for  RED  but  which  will 

not  be  solved  by  any  kind  of  R60,  very  often  because  the  problem 

stems  from  management  policies.  If  the  researcher  can't  do  anything 
about  them,  he  shouldn't  get  involved. 

The  third  thing  is  that,  given  that  you  have  an  operationally  im- 
portant problem  and  a feasible  R&D  approach  to  it,  the  user  has  got 
to  have  a valid  notion  of  what  the  product  of  the  research  is  going 
to  be.  There  has  to  be  agreement  very  early  in  the  game,  perhaps  not 
at  the  proposal  stage,  but  at  least  in  the  problem  definition  stage, 
where  the  researcher  says,  "Okay,  we  understand  what  the  problem  is 
and  we  think  we  can  produce  a solution.  If  we  are  successful  in 
doing  this,  will  you  really  use  it?"  So  there  has  to  be  some 
anticipation  or  even  commitment  on  the  part  of  the  user  that  he  is 
going  to  adopt  that  solution  if  it  is  successful. 

And  the  final  requirement,  which  I have  heard  mentioned  only  by  the 
Army,  is  that  many  users  do  not  recognize  that  they  have  to  contribute 
resources  in  order  to  get  the  R6D  done.  If  you  talk  with  them  in 
the  very  beginning  of  the  project  and  you  say,  "I  think  we  can  solve 
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that  problem  if  you  can  come  up  with  such  and  so,"  and  they  say, 

"Great,  go  to  i t , we  wi 1 1 support  you,"  then  you  are  off  to  a good 
start.  The  researchers  can  go  to  work  developing  a sound  experimental 
design  and  come  back  six  months  later  and  say  we  want  to  test  7.000 
of  your  troops,  or  we  want  to  intervene  in  this  training  process,  and 
so  on.  With  the  kind  of  initial  user  anticipation  and  commitment  I 
have  referred  to,  the  user  is  less  likely  to  say:  "We  never  heard 
about  that."  He  has  to  know  in  advance  what  he  is  going  to  contribute. 

None  of  this  happens  at  the  management  level,  it  all  happens  at  the 
level  of  working  researchers  and  the  immediate  impact  operational 
users.  And  if  any  change  in  the  research  utilization  process  is 
going  to  have  a real  effect,  those  changes  have  to  occur  at  that 
level . 

COMMENT:  Somehow  or  another  the  researcher  does  not  seem  to  play  a 
role  in  this.  Who  is  to  decide  whether  a given  problem  is  trivial 
or  important  for  R&D  investment?  Very  often  it  is  the  researcher 
who  comes  up  with  a plan  or  an  idea  that  at  a given  moment  may  seem 
trivial  to  those  who  are  to  be  the  users  because  they  can't  see  its 
implications.  If  a researcher  would  give  up  at  that  point  and  only 
accept  those  problems  that  are  given  to  him  by  the  user,  there  is  a 
question  as  to  whether  we  would  be  doing  any  significant  research. 

The  researcher  sometimes  may  have  a brilliant  insight  into  the 
utility  of  a piece  of  research--an  insight  that  is  not  shared  by 
the  potential  user.  The  process  should  be  able  to  accommodate 
consideration  of  such  ideas,  because  one  of  the  purposes  of  research 
is  to  be  creative  and  develop  hypotheses  that  others  have  not  thought 
about,  particularly  in  the  6.2  area. 

CAPT  CLARKIN:  When  the  service  and  the  R&D  community  were  giving 
their  separate  reports,  there  was  discontent  on  the  faces  of  both. 

If  you  listen  to  what  was  being  said,  it  was  a cry  for  help,  a cry 
for  assistance  that  the  world  of  technology  and  science  can  provide 
to  them.  There  is  a need  for  interdependence,  a lessening  of  the 
isolation  of  the  several  communities,  all  of  whom  view  each  other 
pejoratively.  We  must  recognize  the  need  for  autonomy  within  the 
R6D  community  just  as  one  must  recognize  the  need  for  autonomy  within 
the  operational  community,  but  we  have  to  modify  that  by  the  process 
of  negotiation  and  compromise  to  be  a responsible  autonomy.  My  hope 
is  that  we  are  going  to  recognize  that  we  are  not  only  distant  from 
the  operational  community  geographically,  but  we  have  a massive  techno- 
logical distance  and  that  distance  can  only  be  overcome  by  interaction, 
interdependence,  communication,  representation,  sens i t i vi ty--and  I 
don't  mean  just  in  a philosophical  context,  I mean  largely  in  the 
marketing  context  as  it  was  addressed  before. 

Most  of  what  I have  heard  was  in  the  area  of  improvements  in  form. 

I believe  the  profit  is  to  be  made  with  one-on-one  interaction,  rather 
than  changes  in  form. 
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OR.  GLASER:  A key  point  in  the  literature  on  research  utilization 
is  that  if  you  want  to  achieve  change,  if  you  want  to  get  anywhere 
with  spread  of  an  innovation,  the  single  most  important  ingredient 
is  interaction  between  researchers  and  all  the  other  stakeholders 
or  gatekeepers  who  can  influence  utilization. 

I think  part  of  the  discussion  here  has  been  on  the  question  of  how 
can  we--by  interpersonal  contact,  by  focusing  on  common  goals,  by 
integrating  the  contributions  of  the  user,  the  researcher,  and  all 
the  people  in-between  and  at  various  levels--do  a still  better  job 
than  we  have  been  doing  at  solving  significant  problems  through  R&D 
and  better  utilization  of  promising  findings  or  products.  Perhaps 
as  we  look  back  each  year  and  count  up  our  score  on  what  we  have 
done  for  the  money  that  has  been  spent  to  support  R6D  contributions 
toward  the  solution  of  significant  problems  as  well  as  toward 
significant  enhancement  of  knowledge  through  basic  research,  we  will 
find  that  we  are  doing  more  than  perhaps  Dr.  Rostker  recognizes. 

And  I think  if  he  takes  a look  at  this  scoreboard  from  year  to  year 
and  gives  it  time,  he  will  find  more  progress  is  being  made  than  is 
apparent  on  the  surface. 

DR.  SANDS:  In  terms  of  attempting  to  assess  from  my  own  subjective 
point  of  view  what  it  is  we  have  done  here,  and  the  importance  of  it, 
I don't  have  a comment  for  you  at  this  time.  I would  like  to  say 
that  I was  delighted  to  have  as  many  of  you  attend  and  participate 
very  actively  as  you  did,  and  at  the  very  beginning  I think  we 
indicated  that  the  challenge  of  how  to  improve  research  utilization 
is  at  hand.  It  still  is  at  hand,  I think.  Out  of  the  proceedings, 
out  of  the  major  conclusions  that  will  be  extracted  from  the  tapes 
we  have  taken  here,  the  extractions  of  literary  guidance  that  we 
received  from  Bert  Griffis,  the  discussions  we  have  had  from 
Admiral  Metcalf,  Captain  Clarkin,  Dr.  Rostker  and  the  faculty  that 
participated  very  actively,  I'm  hoping  that  we  will  at  least  be  able 
to  extract  the  essence  of  hope.  I would  like  to  believe  that  all 
of  us  will  carry  away  from  this  meeting  somewhat  of  a dedication 
to  do  all  we  can  to  begin  to  build  more  important  linkages  and 
establish  many  of  the  types  of  channels  of  communication  that  were 
indicated.  We  need  to  do  this  in  the  interest  of  improving  research 
utilization.  With  that  1 would  like  to  thank  all  of  you. 
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SYNTHESIS 


In  the  course  of  the  three-and-a-ha 1 f day  1977  National  Symposium  on 
Ut i ) i zat ion  of  People-Related  Research,  Development,  Test  and  Evalua- 
tion (RDTSE)  a great  many  obstacles  to  the  utilization  of  human  re- 
sources RsD  were  identified  and  many  recommendations  for  improved 
utilization  were  expressed  by  the  participants.  Some  were  voiced  by 
one  person,  others  by  several,  and  some  appeared  to  represent  a 
general  consensus.  Often  conflicting  views  were  expressed.  The 
following  distillation  represents  an  effort  to  capture  the  essence 
of  the  recommendations.  The  recommendations  are  organized  into  seven 
major  categories.  Inevitably,  the  categories  are  not  mutually  ex- 
clusive, and  some  duplication  of  ideas  may  occur  between  categories. 

COMMITMENT  TO  IMPROVED  UTILIZATION 

There  is  a need  for  serious  commitment  to  the  area  of  research  utili- 
zation. This  commitment  must  be  based  not  only  on  the  conviction 
that  there  is  a critical  need  for  improved  utilization  but  on  the 
recognition  that  the  problems  are  long-standing  and  complex,  and  are 
not  likely  to  be  amenable  to  simple  or  standardized  solutions.  The 
problem  of  utilization  needs  to  be  addressed  in  the  context  of  the 
whole  process  of  program  development  and  resource  development.  It 
needs  to  be  conceptualized  from  a number  of  different  perspectives, 
since  different  approaches  are  likely  to  be  required  for  different  R&D 
program  categories,  different  types  of  consumers,  different  kinds  of 
products,  and  so  forth.  No  single  prescription,  such  as  "do  better 
research",  will  solve  the  problem. 

This  commitment  must  embrace  policy,  planning,  resources,  accounta- 
bility, the  need  for  mechanisms  to  institutionalize  the  process  of 
utilization,  and  the  dedication  of  people.  More  "horsepower"  must 
be  applied  to  the  applications  effort.  Thus,  for  example,  cognizance 
for  human  resources  RDT&E  utilization  should  be  assigned  at  high 
levels  in  the  appropriate  agencies,  funding  and  staffing  should  be 
increased,  and  utilization  should  be  regarded  as  a function  that 
warrants  full-time,  trained  specialists.  Institutional  mechanisms 
for  facilitating  utilization  must  be  developed  and  supported  with 
resources  and  implementing  policy.  Finally,  support  should  be  allo- 
cated for  additional  research  on  research  utilization  in  order  to 
enhance  our  understanding  of  principles  of  effective  utilization. 

INTERACTION  BETWEEN  RESEARCH  AND  USER  COMMUNITIES 

At  present  the  R&O  and  user  communities  tend  to  be  aloof  and  dis- 
dainful of  each  other.  One  of  the  key  means  to  improve  research 
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utilization  will  be  to  find  ways  to  lessen  the  isolation  of  the  re- 
spective communities  while  at  the  same  time  allowing  each  to  maintain 
a responsible  autonomy.  Personal  contact  is  critical  to  improved  RU. 

One  important  mechanism  for  "demilitarizing"  the  boundary  between  the 
research  and  user  communities  is  mutual  education.  There  is  a great 
need  for  cross-education  of  researchers  and  users  as  part  of  an  inten- 
sive process  of  getting  acquainted.  For  example,  users  need  to  be 
better  informed  about  the  R6D  process  (including  the  distinctions 
among  research,  exploratory  development,  advanced  development,  systems 
design,  systems  analysis,  etc.).  They  also  need  to  be  aware  of  how 
they  can  influence  the  programming  of  R&D  resources.  Sponsors  need  a 
better  understanding  as  to  what  are  reasonable  expectations  for  human 
resources  R&O  products,  the  proportions  of  investment  that  are  likely 
to  pay  off  in  the  various  R£D  program  categories,  the  risks  involved 
in  R&D  project  investment,  and  the  time  frames  in  which  to  expect  re- 
sults. Conversely,  the  research  community  needs  to  understand  the 
power  structure  and  the  decision-making  apparatus  of  the  military 
bureaucracy,  and  needs  to  improve  its  understanding  of  the  operational 
environment.  Researchers  also  need  to  be  sensitized  to  the  concept  of 
accountability  and  what  it  may  imply  with  respect  to  the  importance  of 
ut i 1 i zat i on . 

There  was  strong  consensus  that  effective  RU  requires  an  intimate  and 
continuing  process  of  interaction  between  producer  and  user.  For  this 
to  happen,  more  attention  must  be  devoted  to  ways  of  facilitating  in- 
formal, one-on-one  interaction,  and  there  must  also  be  formal  mechanisms 
for  creating  early,  active  and  sustained  involvement  of  potential  users 
in  the  R&D  process.  This  interaction  must  begin  early  enough  to  ensure 
a common  understanding  of  the  R&D  requirement  at  the  inception  of  the 
project  and  continue  long  enough  to  ensure  a smooth  transition  of  the 
product  to  operational  use. 

Various  strategies  should  be  attempted  for  improving  communication 
between  researchers  and  users.  For  example,  a formal  utilization  plan 
should  be  jointly  developed  by  an  ad  hoc  team  of  research  and  user 
representatives  for  each  research  project.  Throughout  the  R&D  process, 
a strenuous  effort  should  be  made  to  make  the  potential  user  a believer 
in  the  product  through  participation  on  advisory  committees,  site 
visits,  briefings,  and  formal  and  informal  project  reviews. 

Technical  assistance  by  the  research  community  to  the  user  community 
in  the  implementation  of  R&D  products  is  another  utilization  strategy 
that  should  be  more  fully  exploited.  Particular  attention  needs  to  be 
devoted  to  mechanisms  for  providing  such  assistance,  and  to  logistics 
and  funding  implications.  Another  strategy  that  has  proven  useful  for 
promoting  closer  relationships  between  research  activities  and  customer 
agencies  is  collocation  of  field  detachments  with  research  laboratories 
and/or  R&D  representatives  or  units  with  operating  commands. 
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RESPONSIBILITY  FOR  FUNCTIONS  AFFECTING  UTILIZATION 


Responsibility  for  the  utilization  function  has  typically  been  dis- 
tributed to  everyone,  and  consequently  has  often  been  discharged 
well  by  no  one.  While  in  a general  sense  utilization  is  everyone's 
job,  in  a more  particular  sense  an  effort  should  be  made  to  allocate 
responsibility  in  such  a way  as  to  make  use  of  the  special ized  com- 
petence of  each  of  the  participants. 

There  is  need  for  a program  to  establish  human  and  organizational 
links  between  research  and  practice.  This  implies  the  establishment 
of  an  explicit  applications  function  within  research  and  user  organi- 
zations and  the  programming  of  special  billets  for  utilization  spe- 
cialists, who  would  serve  as  linking  agents  between  producers  and 
users . 


R&D  REQUIREMENTS 

One  of  the  principal  factors  that  may  influence  the  degree  to  which 
research  products  are  utilized  is  the  extent  to  which  they  address 
operational  requirements  and  policy  issues.  This  issue  was  one  of 
the  major  bones  of  contention  among  the  symposium  participants. 

Dr.  Rostker  contended  that  much  RSD  fails  to  meet  this  test  and  is  con- 
sequently neither  relevant  nor  useful,  and  concluded  that  domination 
of  the  decision-making  process  in  the  planning,  programming  and  bud- 
geting of  applied  R&D  should  be  taken  out  of  the  hands  of  researchers 
and  transferred  to  customers  at  the  policy-making  level.  User  repre- 
sentatives tended  to  endorse  the  view  that  researchers  want  to  "do 
their  own  thing"  and  are  unresponsive  to  user  needs.  Researchers,  on 
the  other  hand,  tended  to  feel  that  a high  proportion  of  R&D  products 
are  useful,  that  often  the  expectations  and  stated  requirements  of  the 
user  are  inappropriate,  and  that  customer  control  of  the  R&D  process 
would  stultify  it. 

There  did  appear  to  be  some  consensus,  however,  that  the  process  of 
formulating  adequate  R&D  requirements  is  complex  and  requires  ongoing 
dialogue  between  users  and  producers.  Further,  there  is  need  for  more 
effective  procedures  for  eliciting,  sharpening  and  monitoring  customer 
requirements.  By  the  same  token,  research  organizations  must  be 
responsive  and  flexible  enough  in  their  work  programs  to  be  able  to 
meet  the  short-term  and  long-term  needs  of  the  customer,  through  a 
spectrum  of  activities  from  technology  base  research  to  technical 
assistance,  and  from  pilot  studies  to  long-term  programmatic  research. 

UTILIZATION  PLANNING 

RADM  Metcalf  pointed  to  the  failure  to  consistently  and  effectively 
plan  for  i mp 1 ementat ior  s the  essence  of  the  utilization  problem. 

There  is  a need  for  better  methods  of  managing  R&D  planning,  including 
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the  explicit  assignment  of  responsibility  for  implementation  planning. 
The  process  of  planning  for  the  utilization  of  R&D  products  needs  to 
be  systematized  and  employed  consistently.  Every  R&D  proposal  should 
include  a utilization  plan,  and  that  plan  should  be  reviewed  and 
iterated  frequently.  The  plan  should  be  as  comprehensive  and  defini- 
tive as  possible,  specifying  such  things  as:  the  user  requirements  to 
which  the  project  is  responsive,  anticipated  products,  expected  appli- 
cations and  implications  of  the  projected  product  or  findings,  poten- 
tial users  and  strategies  for  reaching  them  and  eliciting  their  inter- 
est and  participation,  logistics  support  requirements,  evaluation  pro- 
cedures, implementation  steps,  plans  for  rendering  technical  assistance 
to  aid  the  adoption  process,  costs,  benefits,  risks,  alternate  solu- 
tions and  assignment  of  responsibilities  for  the  various  functions. 

In  short,  there  must  be  early  and  continual  joint  planning  for  utili- 
zation. 


ACCOUNTABILITY 

There  have  been  a series  of  major  factors  or  events  (such  as  the  April 
1977  GAO  report)  that  have  intensified  pressure  for  accountability 
with  respect  to  utilization  of  people-related  research.  This  suggests 
the  need  for  systematic  and  critical  review  and  assessment  of  projects 
and  their  products.  Such  reviews  should  be  conducted  both  on  an  in- 
house  basis  and  through  the  ad  hoc  participation  of  appropriate  user 
representatives.  It  also  suggests  that  the  application  of  cost- 
benefit  analysis  to  R6D  activities  should  become  standard  operating 
procedure. 

Effort  should  be  devoted  to  the  development  of  meaningful  criteria  for 
measuring  the  success  of  an  R6D  activity  as  well  as  the  success  of  its 
utilization  component.  For  a given  project,  early  agreement  should  be 
reached  between  researchers  and  users  as  to  what  criteria  will  be  used 
for  evaluating  the  product  and  assessing  its  impact.  Evaluation  of 
R6D  products  should  be  conducted  and  reported  by  users  as  well  as  re- 
searchers . 

Finally,  the  pressure  for  increased  accountability  suggests  the  need 
for  a coherent  system  for  tracking  and  monitoring  of  R&D  utilization. 

REPORTING  AND  DISSEMINATION 

While  a technical  report  is  not  an  adequate  end  product  in  most  in- 
stances, and  disseminating  reports  does  not  equate  to  utilization, 
nevertheless,  careful  attention  to  presentation  of  results  and  a 
strong  focus  on  dissemination  of  usable  results  contribute  signi- 
ficantly to  the  prospect  that  a product  will  be  utilized.  A quality 
report  requires  time  and  effort,  particularly  if  it  is  to  be  readily 
understandable  to  the  target  audiences.  The  report  should  contain 
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an  executive  summary  that  includes  explicit  implications  and  rec- 
ommendations for  application  or  action,  identifies  categories  of 
potential  users,  and  conveys  cost-benefit  information.  Inviting 
potential  users  to  review  the  draft  report  is  likely  not  only  to 
improve  the  report  but  to  enhance  their  interest  in  utilizing  the 
product.  Similarly,  user  participation  in  briefings  is  often  an 
effective  marketing  tactic. 

Attention  should  also  be  paid  to  transformation  activities,  such  as 
the  translation  of  technical  reports  into  practitioner  documents  and 
the  preparation  of  abstracts  and  research  briefs.  The  use  of  brief 
summaries  of  research  reports,  issued  in  advance  of  the  final  report, 
is  one  mechanism  for  making  results  more  timely  to  meet  user  require- 
ments. 
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San  Diego,  CA  92152 

Mr. 

PH: 

J.  E.  Garside 
(501)  485-4642 

Naval  Ocean  Research  and  Development  Activity 
Bay  St.  Louis,  MS  39529 

Dr. 

PH: 

W.  Githins 
(714)  225-2408 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Dr. 

PH: 

Edward  Glaser 
(213)  879-1373 

Human  Interaction  Research  Institute 

Los  Angeles,  CA  90024 

Mr. 

PH: 

G.  A.  Gimber 
(717)  441-3121 

Naval  Air  Development  Center 

Warminster,  PA  18974 

Dr. 

PH: 

Paul  Greenberg 
(213)  879-1373 

Human  Interaction  Research  Institute 

Los  Angeles,  CA  90024 

Mr. 

PH: 

Bertram  Griffis 
(202)  245-0594 

Rehabilitation  Services  Administration 
Washington,  D.C.  20201 

COL 

PH: 

M.  E.  Grunzke,  USAF 
(205)  279-4252 

Extension  Course  Institute 

Gunther  Air  Force  Base,  AL  36118 

Dr. 

PH: 

D.  G.  Hamreus 
(714)  271-4300 

United  States  International  University 

San  Diego,  CA  92131 

CDR 

PH: 

R.  S.  Hardy,  USN 
(714)  235-3195 

Naval  Manpower  & Material  Analysis  Command, 
Pacific 

San  Diego,  CA  92132 

Mr. 

PH: 

J.  Harris,  Jr. 

(904)  452-1391 

Naval  Education  and  Trai.aing  Support 

Ellyson  Field,  Pensacola,  FL  32509 

Mr . 
PH: 

R.  Harrison 
(703)  224-4270 

Office  of  Civilian  Personnel 

Washington,  D.C.  20390 

Dr. 

PH: 

R.  G.  Havelock 
(313)  764-2560 

University  of  Michigan 

Ann  Arbor,  MI  48106 

Mr. 

PH: 

A.  W.  Himes 
(202)  692-2766 

Director  of  Naval  Laboratories,  NAVMAT 
Washington,  D.C.  20360 
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COL  R.  S.  Hoggatt,  USAF 
PH:  (512)  536-3605 


Applications  Office,  AFHRL 
Brooks  Air  Force  Base,  TX  78235 
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Mr.  Eugene  Hooprich  NPRDC/CNET  Liaison,  NAS  Pensacola 

PH:  (904)  452-2621  Pensacola,  FL  32508 


Dr.  Kay  Inaba 


PH: 

(213) 

883-8200 

LCOL  E.  J. 

Jacko,  USAF 

PH: 

(512) 

671-2794 

Mr . 

Cecil 

D.  Johnson 

PH: 

(202) 

274-8705 

Dr. 

L.  C. 

Johnson 

PH: 

(714) 

225-6671 

Dr. 

James 

Jolly 

Mr. 

Allan 

Jones 

Dr. 

K.  E. 

Kanter 

PH: 

(212) 

797-6024 

Mr. 

William  Keating 

PH: 

(805) 

982-4483 

Dr. 

Norman 

Kerr 

PH: 

(901) 

872-5593 

Mr . 

L.  Klein 

PH: 

(202) 

394-1757 

Mr. 

Rod  Kraatz 

PH: 

(714) 

225-5847 

Mr. 

T.  Krai 

PH: 

(714) 

225-3412 

Mr. 

Michael  Letsky 

PH: 

(202) 

694-3414 

Mr . 

W.  G. 

Lewis 

PH: 

(714) 

225-7532 

Mr. 

George 

Llnsteadt 

PH: 

(714) 

939-7325 

XYZYX  Information  Corporation 
21116  Van  Owen  Street 
Canoga  Park,  CA  91303 
3307  Schools  Squadron  (TAC) 

Lackland  Air  Force  Base,  TX  78236 

Army  Research  Institute  for  Behavioral 
& Social  Science 

AMC  Building,  Alexandria,  VA  22333 

National  Health  Research  Center 
San  Diego,  CA  92152 

School  of  Business  and  Administration 
California  State  University 
Sacramento,  CA  95819 

Naval  Alcoholic  Rehabilitation  Center 
Long  Beach,  CA  90822 

The  Brookings  Institute 
1775  Mass.  Ave.,  N.W. 

Washington,  D.C.  20036 

Naval  Ship  Weapons  System  Engineering  Center 
Port  Hueneme,  CA  93043 

Chief  Naval  Technical  Training  Command 
NAS  Memphis,  Millington,  TN  38054 

Naval  Surface  Weapons  Center,  F'Jhite  Oak 
Silver  Spring,  MD  21090 

Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

Commander  Training  Command,  Pacific 
San  Diego,  CA  92147 

Deputy  Chief  of  Naval  Operations  (0P-112C) 
Washington,  D.C.  20370 

Human  Factors  Div. , Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

Naval  Weapons  Center 
China  Lake,  CA  93555 
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Mr.  John  Lockwood 
PH:  (714)  236-6363 


City  of  San  Diego 
San  Diego,  CA  92101 


Mr.  James  Long 
PH:  (904)  452-4193 

Mr.  W.  F.  Long 
PH:  (A)  436-4417 

Mr.  R.  R.  M,3ckie 
PH:  (805)  968-1071 

Mr.  M.  K.  Malehorn 
PH:  (202)  692-*,836 

Dr.  W.  H.  Maloy 
PH:  (904)  452-2180 

LCDR  L.  S.  Mairs,  USN 

PH:  (202)  694-2278 

Dr.  Warren  Mathews 

PH:  (703)  524-9440  X246 


Dr.  Richard  Mathieu 
PH:  (301)  281-2401 

Dr.  W.  A.  McClelland 
PH:  (703)  549-3611 


Dr.  James  McMichael 
PH:  (714)  225-7122 

RADM  Joseph  Metcalf  III,  USN 
PH:  (202)  694-5626 

Mr.  Charles  F.  Miller 
PH:  (415)  447-1100  X 7191 


Mr.  Gerald  Miller 
PH:  (503)  378-3732 


LTJG  T.  M.  Mitchell,  USN 
PH:  (A)  441-3073 

Dr.  W.  E.  Montague 
PH:  (714)  225-7121 


Chief  of  Naval  Education  and  Training  (N-502) 
Pensacola,  FL  32508 

Naval  Coastal  Systems  Center 
Panama  City,  FL  32401 

Human  Factors  Research,  Inc. 

Goleta,  CA  94302 

Naval  Education  and  Training  (OP-099) 

Washington,  D.C.  20350 

Chief  of  Naval  Education  and  Training,  NAS 
Pensacola,  FL  32508 

Bureau  of  Naval  Personnel  (PERS-21222) 
Washington,  D.C.  20370 

Center  for  Naval  Analysis 
1401  Wilson  Boulevard 
Arlington,  VA  22209 

U.  S.  Naval  Academy 
Annapolis,  MD  21402 

Human  Resources  Research  Organization 
300  N.  Washington  Street 
Alexandria,  VA  22314 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Bureau  of  Naval  Personnel  (PERS-3) 

Washington,  D.C.  20370 

Lawrence  Livermore  Laboratory 
University  of  California 
Livermore,  CA  94550 

Technology  Transfer  Coordinator,  State  of  Oregon 
240  Cottage  Street,  S.E. 

Salem,  OR  97310 

Naval  Air  Development  Center 
Warminster,  PA  18974 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 


Mr.  J.  A.  Murphy 

PH;  (401)  841-3251 

Naval  Undersea  Systems  Center 

Newport,  R.I.  02840 

Mr.  Donald  Nichols 

PH:  (203)  442-0771  X2202 

Naval  Underwater  Systems  Center 

New  London,  CN  06320 

CAPT  S.  L.  Palazzolo,  USN 
PH:  (202)  692-9538 

Naval  Sea  Systems  Command 

Washington,  D.C.  20360 

Mr.  R.  J.  Paszyc 

PH:  (805)  982-5414 

Naval  Civil  Engineering  Laboratory 

Port  Hueneme,  CA  93043 

Dr.  Robert  Penn 

PH:  (714)  225-2252 

Navy  Personnel  Research  and  Development 
San  Diego,  CA  92152 

Center 

CDR  W.  H.  Poe,  USN 

PH:  (714)  235-3195 

Naval  Manpower  and  Material  Analysis  Center, 
Pacific 

San  Diego,  CA  92132 

Mr.  Philip  Postelle 

PH:  (301)  948-0605 

B-K  Dynamics,  Inc. 

15825  Shady  Grove  Road 

Rockville,  MD  20850 

Mr.  W.  H.  Primas,  Jr. 

PH:  (202)  694-2121 

Chief  of  Naval  Operations  (OP-392) 
Washington,  D.C.  20350 

Dr.  James  Regan 

PH:  (714)  225-7106 

Navy  Personnel  Research  and  Development 
San  Diego,  CA  92152 

Center 

Mr . Eugene  Ramras 

PH:  (714)  225-7364 

Navy  Personnel  Research  and  Development 
Dan  Diego,  CA  92152 

Center 

Mr.  B.  D.  Rhea 

PH:  (714)  437-6971 

Operational  Test  and  Evaluation  Force, 
Pacific 

San  Diego,  CA  92135 

LCOL  Ernest  Rider,  USAF 

PH:  (512)  652-3167 

USAF-HDC/DPMYO 

Randolph  Air  Force  Base,  TX  78148 

Mr.  Milton  Ritchie 

PH:  (714)  939-3115 

Naval  Weapons  Center 

China  Lake,  CA  93555 

Dr.  Martin  Rockway 

PH:  (303)  394-2954 

Air  Force  Human  Resources  Laboratory 
Lowery  Air  Force  Base,  CO  80230 

Dr.  Bernard  D.  Rostker 

PH:  (202)  697-2179 

Office  of  Assistant  Secretary  of  the  Navy 
(M&RA) 

Washington,  D.C.  20350 

Mr.  Arnold  Rubinstein 

PH:  (202)  692-2144 

Chief  of  Naval  Material  (MAT-08T244) 
Washington,  D.C.  20360 

Dr.  Franklin  Sands 
PH:  (714)  225-7424 

Dr.  Robert  W.  Sarvis 
PH:  (714)  225-7036 

Mr.  M.  Schwartz 
PH:  (301)  267-2358 


Dr.  Harry  Seymour 
PH:  (202)  432-4760 


Dr.  H.  W.  Sinaiko 
PH:  (202)  381-4277 


Mr.  .Mian  Sjoholm 
PK;  (714)  236-0601 


Mr.  M.  E.  Slawson 
PH:  (714)  235-3195 

Dr.  Robert  Smith 
PH:  (303)  697-1216 

Dr.  Alfred  Smode 
PH;  (305)  646-5212 

Dr.  R.  C.  Sorenson 
PH:  (714)  225-2231 

Dr.  M.  H.  Strub 
PH:  (915)  568-4491 


COL  H.  L.  .Taylor,  USAF 
PH:  (202)  695-9771 

Mr.  G.  J.  Trimble 
PH:  (609)  882-1414 


Mr.  R.  T.  Tyland,  Jr 
PH:  (203)  663-8777 


Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

NPRDC,  Liaison,  David  Taylor  Naval  Ship 
R6D  Center 

Annapolis,  MD  21402 

NPRDC  Washington  Branch  Office 
Washington  Navy  Yard 
Washington,  D.C.  20374 

Smithsonian  Institute 
801  N.  Pitt  Street 
Alexandria,  VA  22314 

San  Diego  Urban  Observatory 

202  "C"  Street 

San  Diego,  CA  92101 

Naval  Manpower  and  Material  Analysis  Center 
San  Diego,  CA  92132 

Chief  of  Naval  Operations  (OP-987H) 

Washington,  D.C.  20350 

Naval  Training  Equipment  Command 
Orlando,  FL  32813 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Army  Research  Institute  for  Behavioral 
& Social  Science 
Fort  Bliss,  TX  79916 

Office  of  Defense  Research  and  Engineering 
Washington,  D.C.  20301 

Naval  Air  Propulsion  Test  Center 
1440  Parkway  Avenue 
Trenton,  N.J.  08628 

Naval  Surface  Weapons  Center 
Dahlgren,  VA  22448 

Army  Research  Institute  for  Behavioral 
& Socia.l  Science 
Alexandria,  VA  22333 
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Dr.  J.  W.  Uhlaner 
PH:  (202)  274-8636 


Mr.  George  R.  Wachold  Pacific  Missile  Test  Center 

PH:  (805)  982-7131  Point  Mugu,  CA  93042 

Mr.  W.  B.  Wagoner  Naval  Air  Test  Center 

PH:  (301)  863-3330  Putuxent  River,  MD  20670 


LCDR  D.  R.  Walsh,  USN  Naval  Manpower  & Material  Analysis  Center 

PH:  (714)  235-3376  San  Diego,  CA  92132 


LCOL  M.  Whaley,  USMC 

PH:  (714)  225-6738 

USMC  Liaison  Office,  Naval  Ocean  Systems 

Center  (033) 

San  Diego,  CA  92152 

Dr. 

PH: 

Charles  Wilson 
(714)  225-2978 

Naval  Ocean  Systems  Center 

San  Diego,  CA  92152 

Mr. 

PH: 

Stanley  M.  Wlnard 
(805)  982-5200 

Naval  Ship  Weapons  Systems  Engineering  Center 
Port  Hueneme,  CA  93043 

Mr. 

PH: 

Dale  Wilson 
(714)  225-7118 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Mr . 
PH: 

Frank  Wolf 
(305)  646-4493 

Naval  Training  Equipment  Center 

Orlando,  FL  32813 

Mr . 
PH: 

Robert  Wolfe 
(301)  267-2635 

David  Taylor  Naval  Ship  R&D  Center 

Annapolis,  MD  21402 

Dr. 

PH: 

M.  M.  Zajkowskl 
(305)  646-5198 

Training  Analysis  and  Evaluation  Group  (TAEG) 
Orlando,  FL  32813 
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APPENDIX  B 


NATIONAL  SYMPOSIUM  OF  THE  MILITARY  SERVICES  ON  UTILIZATION 
OF  PEOPLE-RELATED  RESEARCH,  DEVELOPMENT,  TEST  AND  EVALUATION  (RDT&E) 

AGENDA 


TUESDAY,  14  June  1977 


General  Session — Madrid  Room 


Introduction  to  Symposium 
Administrative  Remarks 


Dr.  Franklin  F.  Sands 
General  Chairman 


Welcoming  Remarks  and  Introduction  CAPT  James  J.  Clarkin 

of  Keynote  Speaker  Commanding  Officer 

Navy  Personnel  R&D 
Center,  San  Diego 

Keynote  Speaker  RADM  Joseph  Metcalf,  III 

Asst  Chief  of  Naval 
Personnel  for  Financial 
Management  & Management 
Information 


RDT&E  Utill^zation:  A Perspective 
from  the  Office  of  DDR&E 


COL  H.  L.  Taylor 
Office  of  the  Director 
Defense  Research  and 
Engineering 


Open  Discussion 

Introduction  of  Faculty;  Dr.  Edward  M.  Glaser 

Group  Interaction  Guidance  Faculty  Ghairman 

Military  Service  Group  Sessions 

Military  service  group  meetings 

Group 

U.S.  Air  Force  & Faculty  Members 
Dr.  Herbert  Clark,  Chairman 

U.S.  Army  & Faculty  Members 
Dr.  Arthur  Drucker,  Chairman 

U.S.  Navy  & Faculty  Members 
Dr.  Robert  Smith,  Chairman 

U.S.  Navy /Marine  Corps  & Faculty  Members 
Dr.  Glenn  Bryan,  Chairman 
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Current  situation  and  needs  assessment  regarding  Research  Utilization  (RU) 
in  the  Army,  Navy,  Air  Force,  and  Marine  Corps.  The  representatives  from  each 
branch  of  the  service  will  break  out  into  their  respective  groups  to  discuss 
current  gaps  and  shortfalls  related  to  cross-validation,  diffusion  and  utiliza- 
tion of  promising  RUT&E  findings.  The  kinds  of  questions  each  group  might 
address  are: 

1.  How  are  decisions  made  by  those  who  administer  RDT&E  funds  regarding 
which  project  or  study  proposals  to  support  from  the  limited  financial 
resources  available? 

2.  Do  we  have  any  data  on  approximately  what  percentage  of  military  RDT&E 
projects  or  studies  turn  out  to  be  of  potentially  significant  value  to 
the  operating  commands  or  offices  of  the  military  services?  What  are 
the  criteria- — liow  do  we  know? 

Wliat  has  been  our  recent  experience  in  attempting  to  use  Human 
Resources  RDT6E  products/findings? 

3.  Of  the  RDT&E  projects  or  studies  that  do  seem  to  have  potential  appli- 
cation, approximately  what  percentage  in  fact  get  implemented  or  tried 
out  by  a significant  number  of  the  various  categories  of  potential  users? 
How  do  we  know?  What  is  the  evidence?  (For  example,  the  Navy  and  Army's 
mess  management  aids  can  be  used  by  (a)  training  activities,  < b)  forces 
in  the  field,  (c)  other  branches  of  the  service,  and  (d)  civilian  indus- 
try through  technology  transfer.) 

A.  Wliat  are  perceived  to  be  the  main  reasons  for  less-than-optimal 

utilization  of  promising  RDT&E  findings,  and  how  might  the  obstacles 
be  reduced? 

What  are  the  barriers  to  research  utilization? 

(a)  In  RDT&E  producers 
•'b)  In  potential  users 

(c)  In  RDT&E  sponsors 

(d)  In  RDT&E  policies,  practices  and/or  policies 

5.  Is  there  significant  opportunity  for  enhancing  the  transfer  of  certain 
military  RDT&E  findings  to  the  private  sector?  If  so,  how  might  this 
be  accomplished? 

6.  Prognosis 

a.  What  can  be  done  in  the  short  run  to  improve  utilization? 

b.  What  long-run  changes  are  necessary  to  improve  utilization? 

c.  What  might  constitute  a useful  model  of  the  utilization  process 
to  guide  us?  Can  we  do  effective  applications  planning?  What 
are  the  critical  factors  Involved  in  applications  planning?  Who 
should  develop  such  a model? 


Plenary  Session — Madrid  Room 


Summary  of  reports  from  each  of  the  military  Discussant: 
services  (Group  Chairman).  Each  report  will  Dr.  Julius  E.  Uhlaner 
be  open  to  questions  of  clarification  on  prob-  Technical  Director 
lems,  needs,  and  opportunities  for  improving  ARI  & Chief  Psycholo- 
RDT&E  utilization.  gist  U.S.  Army 


Introduction  of  Banquet  Speaker 


Dr.  James  J.  Regan 
Technical  Director 
Navy  Personnel  R&D 
Center,  San  Diego 


Banquet — Madrid  Ballroom 


Speaker : 

Dr.  Bernard  D.  Rostker 
Principal  Deputy  Asst 
Secretary  of  the  Navy 
for  Manpower  & Reserve 
Af  fairs 


WEDNESDAY,  15  June  1977 

Reflections  from  the  Operational  Community  Chairman: 

COL  Whaley 


The  State-of-the-Art  in  research  utilization  Dr.  Edward  Glaser 

within  other  federal  agencies:  strategies 

and  problems,  with  special  linkage  to  the 

problems  identified  the  preceding  day  by  the 

military  services. 

Perspective 

National  Institute  of  Mental  Health  (NIMH)  Dr.  Howard  Davis 

Chief,  Mental  Health 
Services  Development 
Branch 


Department  of  Labor  (DOL) 


Mr.  Judah  Drob 
Chief,  Division  of  R&D 
Utilization 
Employment  & Training 
Administration 


Human  Resources  Research  Organization  (HumRRO) 


Dr.  Meredith  Crawford 
George  Washington 
University 


Rehabilitation  Services  Administration 
Department  of  Health,  Education,  and 
Welfare  (RSA-HEW) 


Mr.  Bertrun  Griffis 
Asst  Chief,  Division 
of  Program  Support 
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Center  for  Research  on  Utilization  of 
Scientific  Knowledge  (CRUSK) ; National 
Institute  of  Education  (NIE) 


General  discussion  on  how  experiences  and 
strategies  of  other  federal  agencies  might 
be  applicable  to  the  military 

Overall  Research  Utilization  Perspective 


Administrative  and  Technical  Guidance  for 
Case  Problem  Analyses  scheduled  for  16  June 

Technology  Transfer  Panel  Discussion,  Open 
Discussion  and  Summary 

Mr.  Allen  Sjoholm,  Moderator 
Science  Advisor 

San  Diego  Technology  Action  Center 

Mr.  Robert  C.  Crawford 

Director  of  Intergovernmental  Programs 

National  Science  Foundation 

Mr.  Del  DelaBarre 

President,  California  Innovation  Group 

Mr.  Charles  F.  Miller 
Technology  Applications 
Lawrence  Livermore  Laboratory 

Mr.  George  Lindsteadt 

Director,  Technology  Transfer  Consortium 

Mr.  Gerald  Miller 

Technology  Transfer  Coordinator 

State  of  Oregon 

Mr.  John  Lockwood 
Deputy  City  Manager 
City  of  San  Diego 


Dr.  Ronald  Havelock 
The  Institute  for 
Social  Research 
The  University  of 
Michigan 

Dr.  Edward  Glaser 


Dr.  Thomas  Backer 
Senior  Staff  Associate 
HIRI 


and 

Dr.  William  McClelland 
President,  Human 
Resources  Research 
Organization 

Dr.  Franklin  F.  Sands 
Dr.  Edward  Glaser 

Dr.  Edward  Glaser 
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Some  Issues 


1.  How  to  improve  the  Technology  Transfer  Planning  Process  in  order  to 
better  match  local  government  environment  with  available  technology. 

2.  How  to  achieve  early  recognition  of  local  needs?  What  are  the  primary 
needs  of  the  cities,  counties,  and  states? 

3.  What  are  some  successful  technology  transfer  examples?  What  have  local 
governments  learned  through  these  examples? 

4.  What  is  the  best  way  to  achieve  a two-way  Technology  Transfer  operation 
between  cities,  states,  and  federal  laboratories?  How  to  improve  the 
exchange  of  Information. 


Open  Discussion/Summary 
THURSDAY.  16  June  1977 


Small  Group  Sessions 


Case  Problem  Analysls/Small  Group  Interaction 


Assemble  into  groups  to  analyze  case  problems  describing  seemingly 
valuable  RDT&E  project  outcomes  that  have  not  been  optimally  utilized.  Break 
into  groups  to  formulate  ideas  about  (1)  what  might  have  been  done  differently 
from  the  beginning  in  each  case  to  facilitate  utilization;  (2)  what  can  be  done 
now.  Each  group  would  work  on  several  cases,  but  the  membership  of  each  group 
would  consist  of  a cross-section  of  the  symposium  participants.  Each  group 
would  have  at  least  one  representative  from  each  branch  of  the  service. 


Plenary  Session — Toledo  Room 

Each  group  will  present  its  findings  to  the 
entire  assemblage.  Discussion  at  this  point 
limited  to  questions  for  clarification. 

"Bridge  Building" 


Dr , Edward  Glaser 


Mr.  Merlin  K.  Malehorn 
Deputy  Director 
Education  & Training 
Programs  Division 
Office  of  the  Director 
Naval  Education  and 
Training 
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"Utilization  of  People-Related  Research  and 
Development:  A Case  History  of  Shipboard 
Facilities  Maintenance  Demonstration  Study" 

Open  discussion  and  analysis  of  group 
findings;  faculty  panel  analysis  and 
summary 

Military  Service  Group  Sessions 
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Sumpoiiiwn  Workplan 


14  June 

1,  Have  participants  from  each  branch  of  the  service  meet  in  separate 
small  groups  (joined  by  faculty  members  and  other  participants  not  attached 
to  any  of  the  military  services)  to  identify  current  gaps,  shortfalls  and 
barriers  related  to  cross-validation,  diffusion  and  utilization  of  promising 
nUT&E  findings . 

2.  Reconvene  in  plenary  session  to  hear  reports  from  each  group,  and  attempt 
an  integrated  summary  of  problems,  needs  and  opportunities  for  improving 
RPT&E  utilization  (RU). 

■'  15  June 

I 

^ 5,  Rear  Observations  about  military  problems  and  opportunities  as  surtmarized 

S'  in  No.  2 above  from  heads  of  research  utilization  branches  in  4-S  other  federal 

agencies — as  well  as  problems  encountered  in  their  particular  situations , how 
they  have  dealt  with  those  problems,  and  with  what  results, 

4.  Relate  all  of  the  above  to  the  literature  regarding  factors  that  inhibit 
or  facilitate  utilization  of  promising  RDT&E. 

5.  Discuss  transfer  of  relevant  findings  and  developments  from  federally 
supported  RDl'&E  to  the  problems  of  cities,  states,  and  the  private  sector. 

I 

6.  Sw'imarize  what  has  been  disausscd/lcarned  for  application  to  military 
research  utilization  needs,  problems  and  opportunities . 

f 

% 

16  June 

7.  Undertake  small  group  analysis  of  case  problems  of  RDT&E  project  outcomes 
that  have  not  been  optimally  utilized.  Forsmlatc  stra  tegies  for  improving 
utilization, 

8.  Reassemble  into  pLcvinry  sciisio)i  to  review  and  antique  strategics  pre- 

; pared  by  each  group. 

17  June 

9.  Invite  each  branch  of  the  military  to  meet  together  to  develop  policy 

I recommendations  for  improvement  of  RU  in  their  branch. 

10.  Convene  ge-eral  session  to  hear,  discuss,  erdtique  and  sharpen  policy 
recommendations. 

\ 11,  Evaluate  Rymposiwn. 


r 


APPENDIX  D 

SYMPOSIUM  STEERING 
COMMITTEE 


^ 

I 

APPENDIX  D 


SYMPOSIUM  STEERING  COMMITTEE 
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APPENDIX  E 


ARMY  CASE  STUDY— THE  VALUE  OF  STEREOSCOPIC 
VIEWING  IN  IMAGE  INTERPRETATION 


[Objective  of  the  research  was  to  assess  the  usefulness  of  stereoscopic 

viewing  within  serial  surveillance  systems  in  terms  of  quality  of  In- 
. formation  obtained  and  the  rate  at  which  it  is  extracted.  Both  tactical 

and  strategic  types  of  interpretations  were  used  in  setting  up  perfor- 
i mance  measures  which  were  administered  to  two  matched  groups  of  image 

' interpreters.  For  each  measure,  stereo  pairs  were  provided  to  one  of 

' the  two  groups  and  nonstereo  photographs  to  the  other.  Data  were 

analyzed  by  comparing  mean  scores  through  t-tests  and  analysis  of 
I variance. 

E 

Stereo  viewing  and  nonstereo  viewing  of  the  tactical  and  strategic 
I photographs  were  found  to  be  equivalent  in  terms  of  the  quality  of 

information  provided  and  confidence  expressed  by  interpreters  in  the 
information  they  extracted.  Accuracy  and  number  of  targets  identified 
under  the  two  methods  of  viewing  were  similar;  that  is,  no  statistically 
significant  differences  were  found. 

This  research  had  been  suggested  by  Army  researchers  after  they  had 
been  called  in  by  the  Air  Force  to  consult  on  a similar  problem. 

Findings  in  the  Army  research  then  suggested  strongly  that  the  value 
of  stereo  viewing  should  not  be  taken  for  granted,  and,  in  fact,  led 
Army  researchers  to  suggest  that  the  need  for  the  stereo  capability 
should  be  clearly  demonstrated  before  new  display  equipment 
with  stereo  capability  is  developed  for  use  of  interpreters  in  de- 
tecting and  identifying  militarily  significant  objects.  Several  other 
research  efforts  by  other  o -ganizations  corroborated  these  findings. 

But  Army  researchers  found  a strong  existing  conviction  in  favor  of 
stereo  and  a system  already  highly  geared  up  for  obtaining  and  inter- 
preting stereo  imagery.  The  research  had  little  impact  on  decisions 
to  build  more  stereo  capability,  despite  consistent  replication  of 
the  above  and  the  conduct  of  several  projects  on  overlapping  imagery 
(.60%  is  needed  for  stereo)  that  indicated  conditions  of  nonoverlap 
(except  the  small  percent  needed  to  ensure  100%  coverage)  to  be 
superior  in  time  to  interpret  and  no  different  in  terms  of  accuracy 
or  completeness  of  interpretation. 

The  practical  aspects  of  stereo  coverage  argue  strongly  against  it. 

In  addition  to  the  extra  30  to  40%  in  time  required  for  stereo 
viewing  by  the  Interpreter  mentioned  above,  there  are  several  severe 
systems  costs  involved.  It  takes  twice  as  many  photographs  (and 
processing)  to  cover  a given  area  using  stereo.  Dollar  cost  as  a 
function  of  the  number  of  systems  using  stereo  is  readily  derived. 

It  also  means  that  twice  as  many  reconnaissance  missions  need  to  be 
flown  (again  one  can  compute  dollar  costs)  and  twice  as  many  planes 
and  lives  risked. 

And  still  systems  users  have  persisted  in  using  stereo. 
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ARMY  CASE  STUDY  - REALTRAIN 


RLALTRAIN  is  an  improved,  low-cost  training  and  evaluation  technique 
^ r use  in  Army  tactical  training  exercises  for  combat  units.  Realistic, 
tvvj-sided  free-play  tactical  training  employing  recognized  principles 
of  learning  is  achieved  through  simulated  combat  engagements. 

Originally  this  research  sought  to  develop  a method  for  evaluating 
individual  tactical  performance  under  simulated  battlefield  conditions. 

It  was  felt  necessary  first  to  construct  job  situations  that  would 
demand  that  a man  act  as  he  would  be  expected  to  act  on  a battlefield. 

It  was,  in  retrospect,  not  surprising  that  the  environment  developed 
for  testing  became  a powerful  vehicle  for  training.  The  rationale 
which  provided  the  basis  for  the  Initial  development  of  methods  for 
simulating  the  combat  environment  with  a high  degree  of  psychological 
fidelity  led  directly  to  the  REALTRAIN  method  for  tactical  training. 

Before  a research  organization  recommends  implementation  of  a new 
training  method  it  is  accepted  practice  to  determine  empirically 
whether  the  new  method  works  and  how  well  it  works  (the  degree  to 
which  training  objectives  are  achieved,  the  nature  of  the  skill 
acquisition  curve)  and  to  compare  the  new  method  with  the  method 
it  was  designed  to  replace.  This  was  not  done  with  REALTRAIN.  The 
decision  was  made  by  the  Training  and  Doctrine  Command  (TRADOC)  to 
implement  the  method  before  the  standard  validation  procedures  had 
been  conducted. 

The  reasons  for  TRADOC 's  decision  were  (1)  the  heavy  cost  of  con- 
ducting an  evaluation  of  a new  unit  training  technique  in  the  field, 

(2)  the  rapid  and  enthusiastic  acceptance  of  the  method  by  troops  and 
commanders,  (3)  the  overwhelming  face  validity,  and  (4)  the  fact 
that  no  technique  for  realistic  tactical  training  had  previously 
existed. 

The  REALTRAIN  training  method  was  implemented  by  a TRADOC  Mobile 
Training  Team  (MTT)  during  the  period  3 November  1975  to  5 March  1976 
at  four  divisional  training  sites  throughout  the  U.S.  Army  Europe 
(USAREUR) . The  implementation  in  USAREUR  afforded  an  opportunity  to 
conduct  research  which  could  be  used  to  improve  tactical  training  and 
evaluation  techniques  further  in  an  engagement  simulation  context, 
specifically  providing  a valuable  empirical  base  and  data  source 
for  the  analysis  of  tactical  performance  by  participants  in  the 
exercise,  participant  and  controller  reactions  to  this  new  method, 
and  the  cost  of  conducting  such  exercises. 

That  implemr- tation  by  TRADOC  involved  a cost  of  several  millions. 
REALTRAIN  is  still  not  being  effectively  utilized  today  for  a number 
of  reasons: 

1)  REALTRAIN  has  entailed  very  significant — even  revolutionary — 
changes  from  the  ways  of  past  training  and  it  was  almost  too 


2.  One  specific  resource  requirement  that  has  given  heartburn 
has  been  the  requirement  for  controllers  who  are  required  for 
conduct  of  an  exercise,  though  controllers  learn  as  much  as 
the  trainees. 

3.  Young,  inexperienced  officers  don't  like  to  conduct  exercises 
and  lose  badly,  as  many  of  them  do,  even  though  the  learning 
experiences  are  invaluable.  By  the  same  token,  to  be  a 
participant  casualty  through  inappropriate  performance  in 
the  exercise  is  also  stigmatic.  To  help  solve  the  young 
officer  problem,  a leader  board  game  has  been  devised  to  get 
the  officers  better  prepared  for  the  exercises. 

4.  Logistical  requirements  are  overwhelming — training  ammo  is 
expensive,  tactical  radios  to  support  control  of  the  exercise 
are  hard  to  get. 

TRADOC  officials  have  been  much  aware  that  initial  successful 
utilization  of  REALTRAIN  may  rest  critically  upon  keeping  researchers 
involved  in  the  handing-off  process  to  help  solve  some  of  the  problems 
of  utilization.  The  purpose  of  a new  program  is  to  do  just  that — 
have  researchers  assist  in  preparing  a method  for  implementation  and 
observe  problems  of  utilization  and  help  make  refinements,  periodi- 
cally return  to  reassess  utilization  procedures. 

Comment:  Normally  the  early  grabbing  of  a research  product  for 
implementation  before  the  completion  of  research  is  considered  in 
the  nature  of  eating  the  bean  sprouts,  intended  for  planting  to  ease 
famine.  In  this  instance,  however,  researchers  did  not  consider 
that  harm  had  been  done  to  the  ultimate  utilization  of  this  product, 
especially  in  view  of  the  fact  that  implementation  aided  research. 
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CASE  PROBLEM  (NAVY)  FOR 
NATIONAL  SYMPOSIUM  OF  THE  MILITARY  SERVICES 
ON  UTILIZATION  OF  PEOPLE-RELATED  RDT&E 


A naval  training  school  that  prepares  enlisted  personnel  through 
a nine-month  course  for  performance  in  a critical  shipboard  rate 
was  experiencing  a sustained  serious  problem  of  a high  incidence 
of  student  failure.  Tests  for  progressive  levels  of  performance 
proficiency  are  (and  always  have  been,  a standard  practice)  ad- 
ministered every  two  weeks  after  the  first  three  months  of  training. 
The  final  exam  is  designed  to  measure  the  level  of  performance 
required  on  shipboard. 

In  years  past,  about  30%  of  those  who  entered  the  school  failed 
to  complete  the  training  and  perform  satisfactorily  on  the  final 
exam.  Since  abolition  of  the  draft,  the  failure  rate  has  risen  to 
55%-60%.  A personnel  research  group  was  asked  by  the  CO  of  the 
school  to  study  the  problem  and  see  if  the  failure  rate  could  be 
significantly  reduced. 

The  research  team  conducted  a comprehensive  comparative  study  of 
those  who  succeeded  and  those  who  failed.  They  compared  the  two 
groups  on  a battery  of  psychological  tests  and  biographical  data. 

They  also  undertook  personal  depth  Interviews  with  those  who  ranked 
in  the  upper  25%  of  the  group  who  passed  the  course,  and  with  the 
lower  25%  of  those  who  failed.  They  interviewed  all  instructors 
for  their  perception  of  the  problem,  and  made  a careful  study  of 
the  curriculum  and  instructional  methods,  as  well  as  procedures 
for  assigning  trainees  to  the  school. 

On  the  basis  of  their  findings,  the  researchers  were  able  to  make 
a number  of  important  suggestions  for  change  and  refinement  of  the 
selection  criteria  for  admission  to  this  school,  for  streamlining 
the  instructional  technology,  and  for  attending  to  several  important 
morale  factors. 

When  all  of  these  recommended  changes  were  instituted,  the  failure 
rate  at  this  school  dropped  to  22%.  By  this  time,  the  research 
team  was  convinced  that,  with  the  improved  selection  and  training 
program  that  had  been  worked  out,  theoretically  no  one  should  fail. 
They  then  instituted  a highly  individualized  study  of  those  who 
were  still  failing.  This  Included  individual  psychological  counsel- 
ing, enlistment  of  the  school's  instructors  in  after-hours  individual 
tutoring,  and  ombudsman- like  follow-up  with  each  student  to  see 
whether  identified  needs  and  problems  were  being  met.  As  a con- 
sequence, the  failure  rate  was  further  reduced  from  22%  to  11%. 

The  research  team  published  the  study  both  as  a technical  report 
and  as  a journal  article.  In  these  reports,  they  concluded  that 
they  had  developed  a very  promising  paradigm  both  for  studying  this 
type  of  problem  and  for  applying  remedial  action.  They  felt  that 
it  certainly  could  be  applied  to  the  other  school  in  the  country 
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that  also  trained  for  this  rate,*  and  that  the  procedures  were 
clearly  worth  cross-validation  efforts  by  other  naval  training 
schools  that  were  experiencing  appreciable  failure  rates.  However, 
a year  after  publication  of  their  report,  there  was  no  evidence 
of  spread,  spinoff  or  spillover  in  connection  with  the  failure 
problems  of  other  training  schools. 

The  research  team  feels  discouraged  about  the  lack  of  utilization 
of  a procedure  which  has  had  dramatically  demonstrated  success  and 
which  would  seem  promising  and  applicable  for  many  other  training 
school  situations. 

If  the  CO  of  the  R&D  laboratory  that  conducted  the  study  wanted  you 
to  serve  as  a utilization  consultant  to  review  this  situation, 
determine  why  there  had  been  no  spread,  spillover  or  spinoff  from 
this  promising  R&D  demonstration,  and  whether  utilization  strategies 
might  not  be  initiated,  what  would  you  do,  and  why — for  what 
reasons? 

a.  What  additional  information  would  you  need? 

b.  What  factors  would  you  inquire  into  as  possibly  relevant 
to  the  question  of  why  no  evidence  of  utilization  of  these 
procedures  elsewhere? 

c.  What  would  you  do  or  recommend  be  considered  now  to  promote 
utilization  of  the  procedures  developed  in  this  study?  Why? 

d.  What  would  you  conclude  from  this  case  problem  that  might 
have  general  implication  for  utilization  of  R&D  products/ 
findings? 


* The  average  failure  rate  at  the  other  school  was  about  38%  at 
the  time  the  school  that  requested  the  study  was  experiencing 
55%-60%. 
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ARMY  CASE  STUDY-ARMOR  SCHOOL 


The  Armor  School,  in  conjunction  with  Naval  Training  Equipment  Center 
(NTEC) , contracted  with  General  Dynamics  to  develop  a Miniature 
Armor  Battlefield  (MAB) . The  MAB  had  radio-controlled  tanks  on  a 
miniature  (6')  field,  with  TV  sensors  in  the  tanks  and  hit/kill  sensors; 
it  was  Intended  for  the  simulation  of  platoon  vs.  platoon  engagements. 
Development  cost  was  approximately  $1  million. 

The  Armor  School  asked  HumRRO  to  evaluate  the  system  and  to  work 
out  training  procedures.  The  research  group  had  been  working  on  a 
simplified  version  of  the  system  which  eliminated  some  problems  en- 
countered with  the  TV  sensors  on  the  General  Dynamics  device.  The 
evaluation  was  performed  with  a 24-hour  field  exercise  as  the  per- 
formance criterion;  the  system  was  shown  to  be  effective  for  training 
tank  crews.  A follow-up  in  Europe  using  commanders'  ratings  as  a 
criterion  resulted  in  higher  ratings  for  the  system  than  for  con- 
ventional training  techniques. 

The  Armor  School  recommended  that  the  MAB  and  another  research  product, 
the  Armor  Combat  Decisions  Game  (CDG) , be  developed  by  TRADOC.  Regu- 
lations were  published  governing  the  use  of  these  training  devices. 

NTEC  was  to  have  action  on  obtaining  the  devices,  in  simplified  form, 
per  recommendations  of  HumRRO.  NTEC  tried,  however,  to  Improve  the 
tank  model  further  but  encountered  problems  with  miniaturization  of 
electronic  components.  A contractor  could  not  be  found  to  build  the 
devices  to  specifications;  a later  attempt  by  the  Training  and  Doctrine 
Command,  Deputy  Chief  of  Staff  for  Training  to  get  the  devices  built 
through  Naval  Training  Device  Center  (NTDC)  failed  as  well. 

At  present  the  CDG  is  in  use  by  the  Canadian  Army  in  their  own 
version,  successfully.  The  MAB  has  not  yet  been  produced  as  a 
training  device.  Another  product,  a map  board  which  was  part  of 
project  RECON  for  Armor  training,  was  given  to  NTDC,  formerly  NTEC, 
for  development.  It  is  currently  confined  in  use  to  the  Ohio 
National  Guard. 
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ARMY  CASE  STUDY— ASVAB 


In  February  1966,  the  Assistant  Secretary  of  Defense  for  Manpower  and 
Reserve  Affairs  requested  research  on  a common  aptitude  battery  that 
could  be  used  by  all  the  services  in  the  high  school  testing  program. 

The  Army  was  designated  lead  service  to  determine  to  what  extent 
the  aptitude  tests  of  the  several  services  were  interchangeable  and 
to  develop  an  appropriate  test  battery.  The  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB) , consisting  of  a common  core  of  abbreviated 
forms  of  tests  found  to  be  interchangeable,  was  a first  product  of 
this  endeavor  and  was  put  into  use  to  test  potential  recruits  in  the 
last  year  of  high  school. 

As  the  original  research  called  only  for  the  development  of  tests  for 
the  high  school  program,  only  those  tests  common  across  all  services 
were  considered.  Thus,  the  service  with  the  smallest  set  drove  the 
system.  From  School  Year  1968/1969  ASVAB  Form  1 was  used  in  high 
school  testing;  in  1972  Forms  2 and  3 were  developed  for  high  school 
testing,  and  they  also  became  the  operational  batteries  of  the  Air  Force 
and  Marine  Corps.  Army  bowed  out  as  Executive  Agent  for  ASVAB  research 
and  Air  Force  took  over. 

In  the  middle  of  1974,  the  ASD  (M&RA)  decided  that  as  of  1 January  1976 
there  would  be  a single  classification  battery,  ASVAB,  to  serve  the 
primary  selection  and  classification  purposes  of  all  the  services, 
as  well  as  for  high  school  testing.  The  new  requirement  represented 
an  important  change  of  concept.  To  produce  one  selection  and  classi- 
fication battery  to  serve  needs  of  all  services,  the  service  with  the 
largest  set  of  requirements  drove  the  system,  and  thus,  a 13-test 
battery  was  necessary. 

The  battery  was  fielded  1 January  1976,  but  with  the  short  lead  time 
available,  it  was  done  with  no  validation,  a fact  which  some  Army 
observers  believe  contributed  heavily  to  the  large  attrition  rates 
in  TRADOC  schools.  In  addition,  norms  have  had  to  be  adjusted  and 
are  still  being  questioned  in  the  Army,  Navy  and  Marine  Corps. 

This  case  study  depicts  the  development  of  a product  for  which  sponsor 
Interest,  enthusiasm  and  impatience  are  factors  that  have  to  be  dealt 
with  to  delay  (rather  than  hasten)  utilization,  so  that  the  product 
will  have  a reasonable  chance  to  be  effective  in  operation. 
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EVALUATION 


APPENDIX  F 


r 


END-OF-SYMPOSIUM  EVALUATION  SHEET  RESPONSES 


Symposium  evaluation  sheets  were  completed  and  submitted  by  98%  (64) 
of  the  attendees  (excluding  host  and  support  personnel)  who  partic- 
ipated in  all  sessions  of  the  symposium.  The  following  table 
displays  the  responses  in  each  of  the  nine  areas  which  were  rated. 
The  scale  ranges  from  five  to  zero  (5  being  the  most  positive 
rating) . The  number  in  each  box  shows  the  number  of  attendees  who 
gave  that  rating  and  the  circle  distinguishes  the  most  frequent 
rating  in  each  area. 
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1.  Symposium  Objectives 
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3 

0 

2.  Organization  of  Symposium 
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3.  Value  of  Small  Group  Sessions 
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16 
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5 
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4.  Value  of  Plenary  Sessions 

6 

21 

7 

2 
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5.  Value  of  RU  Agency  Heads 
Discussion  Session 

5 
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15 

9 

3 
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RU  Literature 
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15 

Q| 

6 

6 
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7.  Value  of  Case  Problem  Session 

5 
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8.  Value  of  Policy  Recommendations 
Session 

3 
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14 

10 

11 

1 

More  than  Met  my  Less  than 

met  my  ex-  expecta-  met  my  ex- 
pectations tions  pectations 

9.  Overall  Rating  of  Symposium 

5 

@ 

B 

B 

2 

_j 

a.  Rating  used  for  question  1. 

b.  Rating  used  for  question  2. 

c.  Rating  used  for  questions  3 through  8. 

Note:  The  number  of  responses  does  not  add  up  to  64  for  each  question  since  not  all 
respcHtlents  answered  all  questions.  r -i 


The  final  item  on  the  evaluation  sheet  was  a request  for  sug- 
gestions for  improvement.  The  following  is  a summary  of  the 
ideas  included  in  these  suggestions. 

Several  concerned  the  overall  climate  of  the  symposium.  Three 
people  stressed  that  the  emphasis  should  have  been  on  the  issues 
involved  rather  than  a defense  of  present  policy  and  denial  of 
the  problem.  One  person  described  his  view  of  the  symposium 
crimate:  "The  attitude  which  seemed  to  prevail  was  characterized 

by  one  individual  who  stated  that  'any  specific  recommendations 
for  change  would  constitute  a violation  of  the  chain  of  command.'" 
With  that  sort  of  mind-set  the  predictable  oureaucratic  outcome 
occurred  (i.e.,  several  platitudes,  a recommendation  for  an  ad- 
ditional report  and  a recommendation  for  further  study) . 

Four  people  suggested  that  there  should  have  been  more  small 
group  interaction.  One  specified  that  this  should  be  uninter- 
rupted (e.g.,  no  floating  members). 

There  seemed  to  be  a general  feeling  that  the  objectives  were 
too  broad  and  therefore  prohibited  closure.  As  one  person  put  it, 
"I  simply  believe  that  the  objective  was  not  achieved  because  prob- 
lem definition  and  explanation  used  so  much  time  that  the  closure 
exercise  was  unobtainable  in  the  time  allowed.  I believe  that  the 
individuals  in  attendance  could  have  been  brought  to  closer  grips 
with  the  basic  issue  had  there  been  time  and  emphasis  on  the 
problem  resolution."  Another  suggested  a "less  ambitious  goal-- 
policy  recommendation  was  beyond  ken  of  most  of  those  present." 

The  message  of  eight  attendees  was  clearly  that  more  users  should 
have  been  involved  since  "they  are  an  essential  element  of  any 
solution  to  the  utilization  problem." 

Three  people  stressed  that  the  symposium  should  have  invited  indi- 
viduals who  have  the  ability  to  implement  not  just  make  recommen- 
dations . 

In  general,  the  suggestions  seemed  to  describe  the  need  for  more 
action  rather  than  theorizing,  and  the  attendees  seemed  to  agree 
that  symposia  are  necessary  to  help  attack  the  RU  problem. 
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